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Abstract  of  “  Large  Deviations  and  Quasipotential  for  finite  state  mean  field  inter¬ 
acting  particle  systems  ”  by  Wei  Wu,  Ph.D.,  Brown  University,  May  2014 

We  study  a  general  class  of  mean  field  interacting  particle  systems  with  a  finite 
state  space.  Particles  evolve  as  exchangeable  jump  Markov  processes,  where  finite 
collections  of  particles  are  allowed  to  change  their  state  simultaneously.  Such  models 
arise  naturally  in  statistical  physics,  queueing  systems  and  communication  networks 
literatures. 

In  the  first  part  of  the  thesis,  we  establish  a  large  deviation  principle  for  the 
empirical  measure  process  for  the  interacting  particle  systems.  The  approach  is 
based  on  a  variational  representation  for  functionals  of  a  Poisson  random  measure. 
Under  an  appropriate  communication  condition,  we  also  prove  a  locally  uniform  large 
deviation  principle.  The  main  novelty  is  that  more  than  one  particle  is  allowed  to 
change  its  state  simultaneously,  and  so  a  standard  approach  to  the  proof  based  on 
change  of  measure  is  not  possible.  Along  the  way,  we  establish  an  LDP  for  jump 
Markov  processes  on  the  simplex,  whose  rates  decay  to  zero  as  they  approach  the 
boundary  of  the  domain.  This  result  may  be  of  independent  interest. 

In  the  second  part  of  the  thesis,  we  focus  on  the  mean  field  interacting  particle 
systems  that  only  admit  single  particle  jumps.  Under  the  assumption  that  there 
exists  a  unique  stationary  measure,  we  construct  a  Markov  chain  approximation  of 
the  quasipotential  function  associated  with  the  equilibrium.  This  is  the  first  example 
of  the  Markov  chain  approximation  for  problems  with  non-quadratic  running  cost 
(but  still  convex  in  the  control),  which  may  also  have  singularities  near  the  boundary. 
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Markovian  particle  systems  on  finite  state  spaces  under  mean-field  interactions  arise 
in  many  different  contexts.  They  may  appear  as  approximations  of  statistical  physics 
models  in  higher  dimensional  lattices  (for  various  type  of  spin  dynamics,  see  [33] ,  and 
references  therein)  and  kinetic  theory  [25] .  Recently,  this  type  of  model  also  appears 
in  modeling  communication  networks  [1],  [21],  [23],  [38],  and  game  theory  [22],  The 
dynamics  of  these  particle  systems  have  the  following  common  features:  a)  particles 
are  exchangeable,  that  is,  their  joint  distribution  is  invariant  under  permutation  of 
their  indices;  b)  at  each  time,  some  group  of  finitely  many  particles  can  switch  their 
state  simultaneously;  c)  the  interaction  between  particles  is  global  but  weak,  in  the 
sense  that  the  jump  rate  (of  each  group  of  particles)  is  a  function  of  the  initial  and 
final  configurations  only  of  the  group  of  particles,  and  the  empirical  measure  of  all 
particles.  The  precise  dynamics  of  the  Markovian  n— particle  system  we  consider  are 
described  in  Section  2.1. 

Due  to  the  exchangeability  assumption,  many  essential  features  of  the  state  of 
the  particle  system  can  be  captured  by  its  empirical  measure,  which  evolves  as  a 
jump  Markov  process  on  (a  sublattice  of)  the  unit  simplex.  Under  mild  assumptions 
on  the  jump  rates,  standard  results  on  jump  Markov  processes  (see  [34])  show  that 
the  functional  law  of  large  numbers  limit  of  the  sequence  of  n— particle  empirical 
measures  is  the  solution  to  a  nonlinear  ordinary  differential  equation  (ODE)  on  the 
unit  simplex.  The  ODE  also  characterizes  the  transition  probabilities  of  a  certain 
“nonlinear  Markov  process”  [26]  that  describes  the  limiting  distribution  of  a  typical 
particle  in  the  system,  as  the  number  of  particles  goes  to  infinity,  and  is  commonly 
referred  to  as  the  McKean- Vlasov  limit.  The  first  question  we  address  is  the  sample 
path  large  deviation  properties  of  the  sequence  of  empirical  measure  processes  as  the 
number  of  particles  tends  to  infinity.  In  the  case  of  interacting  diffusion  processes, 
such  an  LDP  was  first  established  by  Dawson  and  Gartner  in  [9].  The  sample  path 
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large  deviation  principle  over  finite  time  intervals  has  a  number  of  applications, 
including  the  study  of  metastability  properties  (via  Freidlin-Wentzell  theory  [20]), 
or  to  quantify  the  fact  that  a  Gibbs  measure  may  evolve  into  a  non-Gibbs  measure 
(i.e.,  no  reasonable  Hamiltonian  can  be  defined)  under  stochastic  dynamics  (which 
is  called  the  Gibbs-non  Gibbs  transtions  in  [39]). 

Large  deviations  principles  for  jump  Markov  processes  are  known  if  the  jump 
rates  are  Lipschitz  continuous  and  uniformly  bounded  away  from  zero  (cf.  [36]). 
In  this  case,  the  large  deviation  rate  function  admits  an  integral  representation  in 
terms  of  a  so-called  local  rate  function.  However,  this  condition  is  not  satisfied  by 
our  model.  Specifically,  as  the  empirical  measure  approaches  the  boundary  of  the 
simplex,  its  jump  rates  along  certain  directions  converge  to  zero.  Nevertheless,  we 
show  that  (under  general  conditions  on  the  jump  rates),  the  sequence  of  empirical 
measure  processes  satisfies  a  sample  path  LDP  with  the  rate  function  having  the 
standard  integral  representation.  Linder  mild  conditions,  we  also  establish  a  “locally 
uniform”  refinement  [36],  which  characterize  the  decay  rate  of  the  probabilities  of 
hitting  a  convergent  sequence  of  points.  Such  result  is  of  relevance  only  for  discrete 
Markov  processes  (and  not  for  diffusions)  and  does  not  follow  immediately  from  the 
LDP.  Indeed,  we  provide  an  example  (Example  3.1.26  in  Section  3.1.4)  where  the 
sample  path  LDP  holds,  but  its  locally  uniform  refinement  does  not.  The  locally 
uniform  refinement  is  shown  in  [7]  to  be  relevant  for  the  study  of  stability  properties 
of  the  nonlinear  ODE  that  describes  the  law  of  large  numbers  (LLN)  limit.  All  the 
main  results  of  this  paper  are  formulated  for  a  more  general  class  of  jump  Markov 
processes  on  the  simplex  whose  rates  diminish  to  zero  at  the  boundary,  and  the 
interacting  particle  models  are  obtained  as  a  special  case. 

Other  works  that  have  studied  large  deviations  for  jump  Markov  processes  with 
vanishing  rates  include  [35] ,  [30]  and  [3] .  However,  the  results  in  [35]  impose  special 
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conditions  on  the  jump  rates  near  the  boundary,  which  do  not  apply  to  our  model  (see 
Appendix  A).  On  the  other  hand,  the  method  in  [30]  and  [3]  are  adaptations  of  the 
argument  used  by  Dawson  and  Gartner  in  [9],  which  crucially  relies  on  the  fact  that 
the  measure  on  path  space  induced  by  the  interacting  n— particle  process  is  absolutely 
continuous  with  respect  to  that  induced  by  n  independent  (non-interacting)  particles, 
each  evolving  according  to  a  time  inhomogenous  Markov  process.  This  property  does 
not  hold  when  multiple  particles  jump  simultaneously.  Simultaneous  jumps  are  a 
common  feature  of  models  used  in  many  applications  (see  Example  2.4.3  and  also 
[38]  and  [18,  Chapter  8]). 

The  large  deviation  upper  bound  follows  from  general  results  in  [13]  (see  Section 
3.4).  The  subtlety  is  to  prove  the  large  deviation  lower  bound.  Our  strategy  for  the 
proof  is  based  on  a  variational  representation  for  the  n— particle  empirical  measure 
process  and  a  perturbation  argument  near  the  boundary.  The  starting  point  of  our 
variational  representation  is  a  representation  formula  for  the  functionals  of  Poisson 
random  measures  [6],  and  an  SDE  representation  of  the  empirical  measure  process 
in  terms  of  a  sequence  of  Poisson  random  measures.  However,  the  state  dependent 
nature  of  the  jump  rates  leads  to  a  somewhat  complicated  variational  problem.  We 
use  the  special  structure  of  the  SDE  to  simplify  the  representation  formula.  The 
perturbation  argument  takes  inspiration  from  [15],  where  an  LDP  was  established 
for  a  discrete  time  one  dimensional  Markov  chain.  Our  model  is  higher  dimensional, 
where  the  perturbation  argument  becomes  substantially  more  intricate,  and  geom¬ 
etry  comes  into  play.  The  variational  representation  that  we  establish  holds  more 
generally  for  jump  Markov  processes  with  bounded  jump  rates,  and  could  be  useful 
for  obtaining  other  asymptotics. 

In  the  situation  when  the  law  of  large  numbers  dynamics  of  the  Markov  process 
has  multiple  attractors  in  a  domain,  the  system  typically  spends  a  long  time  in  a  small 
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neighborhood  of  an  attractor  and  rarely  makes  an  excursion  away  from  it.  After  a 
long  time  and  multiple  excursion  attempts,  a  large  fluctuation  occurs  and  makes  one 
of  these  attempts  succeed,  so  the  system  approach  one  of  the  other  attractors.  Such 
phenomena  of  noise  induced  rare  jumps  between  attractors  is  called  metastability. 

The  work  of  Freidlin  and  Wentzcll  establishes  the  connections  between  large  de¬ 
viation  principle  and  the  metastable  properties  of  the  underlying  Markov  process.  A 
key  notion  in  their  framework  is  the  so  called  quasipotential,  which  has  a  representa¬ 
tion  takes  the  form  of  calculus  of  variations  (as  an  exit  problem).  The  quasipotential 
provides  important  information  that  can  be  used  to  study  transitions  of  the  inter¬ 
acting  particle  systems  between  different  regions  of  the  domain.  It  provides  the 
asymptotic  descriptions  of  the  likelihood  of  a  path  and  the  exit  distribution  from  a 
domain  (at  the  level  of  exponential  rates)  when  such  a  transition  happens. 

In  most  cases,  one  cannot  obtain  an  explicit  solution  for  the  quasipotential,  and 
therefore  a  numerical  approximation  is  needed.  A  standard  method  for  the  construc¬ 
tion  of  approximations  in  optimal  control  problems  is  the  Markov  chain  approxima¬ 
tion  method.  This  method  is  based  on  directly  approximating  the  controlled  process 
that  appear  in  the  variational  problem  by  a  finite  state  controlled  Markov  chain, 
and  define  an  analogous  minimal  cost  function  as  the  approximation.  This  method 
has  been  used  in  the  context  of  [28] ,  [29] ,  [4] ,  [16]  and  so  on,  and  is  especially  useful 
when  the  corresponding  PDE  characterization  (Hamilton-Jacobi-Bcllman  equations) 
is  not  well  defined  or  difficult  to  work  with. 

There  are  many  standard  cost  structure  for  which  the  convergence  of  the  Markov 
chain  approximation  as  the  discretization  goes  to  zero  is  well  known.  However,  our 
cost  structure  is  novel,  in  the  sense  that  the  linear  quadratic  structure  (and  strict 
positivity)  of  the  running  cost  no  longer  hold,  and  the  boundary  of  the  associated 
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exit  problem  is  a  singleton.  We  adapt  the  method  of  [29]  and  [4]  in  our  setting, 
and  in  an  upcoming  work  [17]  we  prove  a  convergence  result  of  the  approximation 
problem. 

The  outline  of  this  paper  is  as  follows.  In  Chapter  2  we  set  up  the  mean  field 
interacting  particle  system,  and  describe  a  few  examples  in  the  literature  that  fit  into 
our  framework.  Chapter  3  is  devoted  to  the  proof  of  the  large  deviation  principle  for 
the  empirical  measure  process  for  such  interacting  particle  systems.  Some  technical 
details  are  deferred  to  Appendix  B  and  C.  In  Chapter  4,  we  restrict  to  the  mean 
field  interacting  particle  system  with  single  particle  jumps.  We  construct  Markov 
chain  approximations  of  the  quasipotential  function  associated  with  the  stationary 
measure  of  such  interacting  particle  systems,  and  give  a  proof  of  convergence.  Finally, 
in  Appendix  A  we  briefly  describe  why  the  conditions  in  [35]  do  not  apply  to  our 
large  deviation  problem. 
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2.1  Model  Description 


Here  we  introduce  the  finite  state  mean-field  interacting  particle  system.  Consider 
an  n-particle  system  in  which  the  state  of  each  individual  particle  takes  values  in 
the  finite  set  X  =  {1,2,  Let  P  (X)  denote  the  space  of  probability  measures 

on  X.  We  identify  P  (X)  with  the  simplex  S  =  {p  G  :  Pi  >  0 ,Y^t=iPi  =  1} 
and  endow  S  with  the  topology  induced  from  Rd,  so  that  P  (X)  is  equipped  with 
the  Euclidean  norm  ||-||.  Define  Pn  (X)  =  {L  Y^i=i  ^  '■  %  £  XnJ  C  P(X),  where  Sx 
represents  the  Dirac  mass  at  x.  Then  Pn  (X)  can  be  similarly  identified  with  the 
lattice  Sn  —  S  f 1  -Zd. 

For  each  i  =  1,  ...,n,  let  Xhn  (t)  be  the  state  of  the  ith  particle  at  time  t.  We  as¬ 
sume  that  the  sequence  of  processes  Xn  (•)  =  {Xn  (t)  =  (X1,n  (t) ,  ...,Xn’n  (t)) ,  t  >  0}, 
n  e  N,  are  defined  on  a  probability  space  (D,  P,  P).  Each  Xn  (•)  evolves  as  a  cadlag, 
Xn- valued  jump  Markov  process.  The  associated  empirical  measure  is  denoted  by 

1  n 

fp  (t,u)  =  -'^2  <,»(*, w) ,  t  >  0,  UJ  G  fl 

n  i= l 

In  subsequent  discussions,  we  will  often  suppress  the  dependence  on  u. 

The  possible  transitions  of  Xn  are  as  follows.  It  is  assumed  that  there  exists 
K  G  N  such  that  at  most  K  particles  jump  simultaneously.  When  K  =  1,  then 
almost  surely  at  most  one  particle  can  instantaneously  change  its  state.  If  the  particle 
changes  its  state  from  i  to  j,  for  some  i,j  G  X,  the  transition  rate  is  assumed  to 
be  T”.  (/in  (f)),  where  (Tn  (x) ,  x  G  5}  is  a  family  of  nonnegative  d  x  d  matrices,  and 
to  be  consistent  with  a  convention  used  when  K  >  1  we  set  T”  (x)  =  0.  In  the 
case  of  general  K ,  for  each  1  <  k  <  K,  an  ordered  collection  of  k  particles  among 
all  possible  ordered  k— tuples  of  the  n— particle  system  can  simultaneously  change 
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its  configuration  from  i  =  (*i,  ..,ik)  G  Xk  to  j  =  (j \ ,  ...,jk)  G  Xk,  where  %i  ^  ji,  for 
l  =  1, k.  Note  that  it  is  possible  that  several  particles  may  be  in  the  same  state. 
Let 

Jk  =  {(i,  j)  G  x  :  n  ±  j,  for  l  =  1, k) 

be  the  collection  of  all  possible  pairs  of  initial  and  final  configurations  for  an  ordered 
k— tuple  of  particles.  At  time  t,  the  rate  of  a  simultaneous  jump  of  a  k— tuple  from 
i  G  Xk  to  j  G  Xk  is  given  by  r|tn  (/jn  (£)),  where  |rfc,n  (x)  ,165}  is  a  family  of 
nonnegative  dk  x  dk  matrices  with  r~n  (x)  =  0  if  (i,  j)  ^  Jk .  We  also  assume  the 
jump  rate  is  independent  of  the  ordering  of  the  particles:  if  Sk  denotes  the  group  of 
permutations  on  {1, ...,  k},  then 

rjj”  ( x )  =  rtjMj)  ( x ) ,  for  any  n  G  N,  k  =  1, ...,  K,  x  G  S  and  a  G  Sk.  (2.1.1) 

2.2  Dynamics  of  the  Empirical  Measure  Process 


If  the  initial  condition  is  exchangeable,  then  it  is  clear  that  the  n— particle  system 
described  above  is  exchangeable,  and  thus  its  state  can  be  described  by  the  empirical 
measure  fin. 

The  empirical  measure  process  {/in  (t)  ,t>  0}  is  a  cadlag  jump  Markov  process 
that  takes  values  in  Sn.  We  now  identify  its  generator  Cn.  Let  {e,,i  =  1  ,...,d} 
represent  the  standard  basis  of  Wl.  When  K  =  1,  the  possible  jump  directions  of 
/in  (•)  lie  in  the  set  ^Vi,  where  V\  =  {ej  —  e,,  (i,j)  G  J1}-  Moreover,  the  number  of 
particles  in  state  i  when  the  empirical  measure  is  equal  to  x  G  Sn  is  nxi.  Hence,  the 
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jump  rate  of  /in  (•)  in  the  direction  2-  (ej  —  ef)  is  nXiT\p  (x),  and  Cn  takes  the  form 

Az  (/)  (x)  =  n  Y  xirijn(x)  f  (x  +  -  (ei  -  e*)')  -  f  (x)  (2.2.1) 

(ij)ej-1  L  V  n  /  J 

for  any  bounded  function  /  :  Sn  i— >•  R. 

In  the  general  case  when  K  e  N,  for  1  <  k  <  K,  i  =  {ii,  G  Tfc,  n  G  N 

and  a;  G  <Sn,  let  y  G  Xn  be  a  collection  of  n  particles  whose  empirical  measure  is  x. 
Define  Ak(n,i,x)  to  be  the  number  of  ordered  subsets  of  {1,  ...,n}  of  size  k,  with 
the  property  that  if  (j i , . . .  ,jk )  is  the  subset,  then  (yn, . . .  ,yjk)  =  (ii, . . .  ,ik ),  he., 
the  Ith  particle  in  the  subset  is  in  state  p.  Then  Ak  (n,i,x)  takes  the  form 

k 

Ak  (n,  i,  x)  =  nk  xk  +  O  (nfe_1)  ,  (2.2.2) 

i=i 

where  the  error  term  is  non-zero  when  the  states  {fy}f=1  are  not  all  distinct. 

For  k  =  1, ...,  K  and  i  G  Xk,  denote  e\  =  Ym= i  eui  and  define  V  =  UfL1  14,  where 

Vk={ei-ei:  (i,  j)  £jk). 

We  call  v  =  ej  —  e\  the  jump  direction  associated  with  (i,  j). 


Lemma  2.2.1.  The  generator  of  the  Markov  process  {/in  (•)}  is  given  by 

£n  (/)  (x)  =  n  Y  Y  aif  f  (x  +  \ei  -  ^ei)  -  /  (x 
k=1  (ij  )&Jk  ^  V  U  U 

for  any  bounded  function  f  :  Sn  h- >•  R,  with 

(x)  =  (n>  b  x)  Tkf  (x),  x  G  Sn. 


(2.2.4) 
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Alternatively ,  the  generator  can  be  written  as 


(/)  (x)  =  n  K  (®) 

v£V 


f  [X  +  -V  -  /  (x) 
n 


where 


K 


k=1  (ij)e  Jk: 

ej  —e\=v 


(2.2.5) 


(2.2.6) 


Proof.  Define  an  equivalence  relation  ~  on  Jk  as  follows:  for  (ii,  ji) ,  (i2,  j2)  £  Jk  1 
(ii ,  j1)  ~  (i2,  j2)  if  and  only  if  there  exists  a  G  Sk  such  that  a  (ii)  =  i2,  cr  (jx)  =  j2. 
Let  [i, j]  denote  the  equivalence  class  containing  (i,j),  [jk]  denote  the  collection 
of  equivalence  classes,  and  dehne  Sk  [i,  j]  =  {a  G  57  :  cr  (i)  =  i,  cr  (j)  =  j}.  Since  the 
particles  are  assumed  indistinguishable,  when  (iiJx)  ~  (i2,j2),  the  jump  associated 
with  (ii,  ji)  coincides  with  the  jump  associated  with  (i2,  j2).  Therefore,  given  (i,  j)  G 
ffk,  the  number  of  distinguishable  ordered  k— tuple  transitions  from  configuration  i 
to  j  is  equal  to  .  By  the  permutation  symmetry  (2.1.1),  we  can  set  T^’?j  (•)  = 

r(j'n  (•),  and  the  generator  of  the  Markov  process  {fin  (•)}  is  given  by 


*=MU|eM  1  *MI  '  J 


iui6[^ 

for  any  bounded  function  /  :  S, 


,  x  G  Sn 

(2.2.7) 


An  alternative  way  is  to  write  the  generator  (2.2.7)  as  a  sum  over  Jk  rather 
than  over  [fTk]  ■  Noting  that  |[i,j]|  =  ^5;  L  =  |g)jj  jii  ,  we  can  rewrite  (2.2.7)  as  in 
(2.2.3).  □ 


We  will  state  and  prove  our  results  in  the  formulation  of  Markov  processes  with 
generator  (2.2.5);  the  precise  implications  for  the  original  n— particle  system  are 
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described  in  Section  3.1.4. 


2.3  The  Law  of  Large  Numbers  Limit 


The  functional  law  of  large  numbers  (LLN)  result  for  general  interacting  jump  pro¬ 
cesses  was  established  in  [34]  (see  also  [27]).  Since  the  properties  of  the  law  of  large 
numbers  trajectory  will  be  used  in  the  large  deviation  proof,  for  completeness  we 
present  a  proof  for  our  interacting  particle  system  (Theorem  2.3.2)  in  Section  3.3.3. 

The  following  condition  is  used  to  obtain  a  unique  law  of  large  numbers  limit. 

Condition  2.3.1.  For  each  v  G  V,  there  is  Lipschitz  continuous  \v  :  S  — >  M  such 
that  A™  (•)  converges  uniformly  to  Xv  (•)  on  S. 

We  also  define 

M  =  sup  A„  ( x )  <  oo.  (2.3.1) 

vGV,x£S 

We  now  state  a  well  known  law  of  large  numbers  result  for  the  sequence  of 
processes  {/in}neN  [34], 

Theorem  2.3.2.  Assume  the  family  of  jump  rates  {A™  (•)  :  v  G  V,n  G  N}  satisfy 
Condition  2.3.1.  Also,  assume  pn  (0)  converges  in  probability  to  p0  G  T’(T)  as  n 
tends  to  infinity.  Then  {pn  (-)}neN  converges  ( uniformly  on  compact  time  intervals) 
in  probability  to  p{-),  where  p{-)  is  the  unique  solution  to  the  nonlinear  Kolmogorov 
forward  equation 

ii(t)  =  y^A „  (p  ( t )) ,  p  (o)  =  /i0. 

oev 


(2.3.2) 
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We  now  describe  the  LLN  limit  for  the  interacting  particle  systems.  We  first  state 
an  assumption  on  the  rates  {Ty  n  (•) ,  (i,  j)  G  Jk,k  =  1  ,...,K,n  G  N}  that  implies 
Condition  2.3.1. 

Condition  2.3.3.  For  every  k  =  1, ...,  K  and  (ij)  G  Jk ,  there  is  Ty  :  S  — >  M 
such  that  for  every  x  G  S  and  sequence  xn  G  Sn,  n  G  N,  such  that  lirn^oo  xn  =  x, 
Ty  (x)  =  lirn^oo  nfc_1ry’n  (xn),  and  the  family  of  rate  matrices  {Ty’n  (•)}  satisfies  the 
permutation  symmetry  (2.1.1).  Also,  each  «y  (•)  is  Lipschitz  continuous,  where 

«y  0)  =  ay ' H  (x)  =  ^[  II  Xi^i  '  (2-3-3) 

i=i 


Given  a  Markov  process  with  generator  (2.2.3),  one  can  always  define  a  single 
particle  jump  process,  with  the  jump  rate  matrix 


ij 


X)  = 


K  k 

XXX 

fc=1  (ij)eyfc  i= i 


k,n 


a ; 


x 


Xi 


x  G  S 


(2.3.4) 


for  (i,j)  G  J1  and  n  G  N.  When  Xi  =  0,  akF  (x)  /xi  is  understood  as  the  pointwise 
limit  of  Oy’n  (y)  /yl3  when  y  lies  in  the  relative  interior  of  S  and  y  x  in  the 
Euclidean  norm.  Here  the  superscript  “eff”  stands  for  “effective,”  and  indicates  that 
the  single  jump  process  will  have  the  same  LLN  limit  (2.3.2).  Indeed,  from  Condition 
2.3.3,  it  is  clear  that  for  each  ig5,  r”’eff  (x)  converges,  as  n  — >  oo,  to 


K 


r*J  M 


E  E  E 

k=1  (ij  )&Jk  1=1 


(  k 

n 

r= 1 

W* 


Xi. 


[X 


3{ i=k,j=jl }■ 


(2.3.5) 


Moreover,  if  Condition  2.3.3  holds,  then  the  jump  rates  {A™}„gV  defined  by  (2.2.6) 
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satisfy  Condition  2.3.1,  with 

K 

=  £  Y,  “lw.  <2-3-6) 

k=1  (ij)e  Jk: 

ej  —e\=v 

However,  as  elaborated  in  Example  3.1.26,  the  large  deviation  behavior  of  these  two 
systems  can  be  quite  different. 


2.4  Examples 


The  particle  systems  that  we  describe  naturally  occur  in  a  wide  range  of  areas,  in¬ 
cluding  statistical  mechanics  (Curie- Weiss  model),  graphical  models  and  algorithms, 
networks  and  queueing  systems  (rerouting,  loss  networks).  We  present  a  few  illus¬ 
trative  examples  below. 

Example  2.4.1.  Opinion  dynamics  of  the  Curie-Weiss  model  [11],  This  is  a  mean 
field  model  on  a  complete  graph.  Let  d  =  2,  and  as  before,  let  n  be  the  number  of 
individuals  and  Xl'n  (t)  e  {  —  1,1}  denote  the  opinion  of  the  ith  individual  at  time 
t.  At  time  0,  each  individual  takes  opinions  Xl'n  (0)  independently  and  uniformly 
at  random.  Each  individual  has  an  i.i.d  Poisson  clock  of  rate  1.  If  the  clock  of 
individual  i  rings  at  time  t,  he/she  computes  the  opinion  imbalance 


=  xj’n  (t-), 

and  changes  opinion  with  probability 


Vmr  (X'-"  (t)) 


exp  (— 2/3|MW|/n)  if  (f — )  >  0 


1 


otherwise. 
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The  empirical  measure  process  \in  only  takes  jumps  of  the  form 
V  =  {ej  —  ei,i,j  G  {  —  1, 1}}.  It  satisfies  Condition  2.3.3  with  T  =  r1,  and 

exp  (-2(3  (/a!  -  fi_ i))  if  p1  -  p._i  >  0 

Tp-ii/i)  =  < 

1  otherwise. 

exp  (-2(3  (/i_i  -  Hi))  if  H l-yU-iCO 

r-i,i  (fi)  =  < 

1  otherwise. 

and  (p)  =  (p)  =  0. 

A  generalization  of  the  above  example  is  the  Curie- Weiss-Potts  model  with 
Glauber  dynamics,  which  we  described  below,  the  mixing  time  of  which  has  in¬ 
teresting  phase  tranistion  properties  (see  [31]). 

Example  2.4.2.  Glauber  dynamics  of  Curie-Weiss-Potts  model.  Fix  n  >  0,  Xl,n 
are  interpreted  as  spins  that  take  value  in  X  =  {1,  ...,d}.  Let  X  =  {A^1,n,  ...,Xn,n}. 
Given  (3  >  0,  the  Curie-Weiss  distribution  is  a  probability  measure  on  Xn ,  given  by 

f  (3  n  n 

un  (X)  =  Zp*ex p - 1(a^"=ao>} 

\  n  i= i  j=i+ 1 

where  Zq.u  is  the  normalizing  constant.  The  continuous  time  Glauber  dynamics 
(Xt)t>0  for  the  spin  flips  is  defined  as  follows.  Each  particle  has  an  i.i.d  Poisson 
clock  of  rate  1  attached  to  it.  Assume  the  clock  at  particle  i  rings  at  time  t,  we  set 
the  new  value  of  Xl,n  to  be 

k  G  X  with  probability  vn  (AT*,n  =  k\X^,n  =  X •7,n  (t—) ,  Vj  ^  i) 


The  empirical  measure  pn  is  interpreted  as  the  magnitization,  which  evolves  as  a 
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Markov  process  with  generator  (2.2.1),  with 


exp  (~|  (nxj  -  1)) 

E&= i  exP  (-£  - !)) 


for  x  E  Sn. 


Therefore,  for  x  E  S, 


lim  Xir^n  (x) 

1 - WYl  J 


^  exp  (-/3xi) 

*ELiexP  (-#**) 


Simultaneous  jump  models  arise  naturally  in  the  context  of  communication  net¬ 
works.  We  now  provide  one  such  example,  from  [21].  Many  more  can  be  found  in 
[38],  [33]  and  [24], 

Example  2.4.3.  Alternative  rerouting  networks  [21].  Consider  a  network  that  con¬ 
sists  of  n  links,  each  with  finite  capacity  C .  Let  X  =  {0, C}  and  let  Xl,n  denote 
the  number  of  customers  using  link  i.  At  each  time  t,  customers  or  packets  arrive 
to  each  link  as  a  Poisson  process  with  rate  A  >  0.  If  a  packet  arrives  to  a  link  with 
spare  capacity,  then  it  is  accepted  to  the  link  and  occupies  one  unit  of  capacity  for  an 
exponentially  distributed  time  with  mean  one.  On  the  other  hand,  if  a  packet  arrives 
to  a  link  that  is  fully  occupied,  two  other  links  are  choosen  uniformly  at  random  from 
amongst  the  remaining  n  —  1  links.  If  both  chosen  links  have  a  unit  of  spare  capacity 
available,  the  packet  occupies  one  unit  of  capacity  on  each  of  the  two  links,  for  two 
independent,  exponential  clocks  with  mean  one.  Otherwise,  the  packet  is  lost.  This 
model  seeks  to  understand  the  impacts  by  allowing  alternative  routes  that  occupy  a 
greater  number  of  resources. 

The  empirical  measure  process  /in  is  a  jump  Markov  process  with  jump  rates 
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summarized  as  follows:  for  any  i,j  E  X: 


Tn  +  £  (e*+i  -  ef) 

+  £  (ei_i  -  e*) 

A4”  +  n  (e*+!  —  e*  +  ej+l  ~~  ej) 
k  ^  |  (ei+l  -  e0 


at  rate  nX/if 
at  rate  nipf 
at  rate  2A 

(n— l)(n— 2) 

at  rote  K"-1) 

(n— l)(n— 2) 


0  <  i  <  C  —  1, 

1  <  i  <  C, 

0  <  i  ^  j  <  C  -  1, 
0  <  i  <  (7  —  1. 


This  model  satisfies  Condition  2.3.3  with  K  =  27  and  with  matrix  entries 


^ijj+l  (aO  —  \  ^i,i- 1  (/^)  —  -^(tj),(j+lJ+l)  (/^)  —  -V^c 

and  zero  otherwise.  By  (2.3.5),  we  can  calculate  the  effective  jump  rate  as 

(aO  —  r ]j  (n)  +  ^  ^  (/i)  +  A^rj^  (aO  , 

i'ft' 

which  gives  rfj+1  (n)  =  X+2  /ii'A/ic+/^A/ic  =  A  [1  +  pc  (2  -  /i,)]7  r|f_1  (/i)  =  A, 
and  rj  (/i)  =  0  otherwise. 
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Large  Deviations  for  Finite  State 
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This  chapter  is  organized  as  follows.  In  Section  3.1  we  state  the  main  result,  which 
is  a  sample  path  LDP  for  a  general  class  of  state  dependent  jump  Markov  process 
(with  rates  that  are  allowed  to  vanish)  as  well  as  its  locally  uniform  refinement.  We 
also  describe  how  the  general  result  specializes  to  the  mean  field  interacting  particle 
systems.  In  Section  3.2  we  verify  the  conditions  of  the  main  results  for  the  empirical 
measure  processes  associated  with  a  large  class  of  interacting  particle  systems.  Sec¬ 
tion  3.3  establishes  the  variational  representation  for  the  empirical  measure  process, 
and  provides  an  alternative  proof  for  the  functional  LLN  limit.  Some  details  of  the 
proof  of  the  variational  representations  are  deferred  to  Appendix  B.  The  sample  path 
large  deviation  upper  and  lower  bounds  are  derived  in  Sections  3.4  and  3.6,  respec¬ 
tively,  while  in  Section  3.5  we  study  properties  of  the  local  rate  function.  Section  3.7 
is  devoted  to  the  proof  of  the  locally  unifom  LDP. 


3.1  Main  Results 

3.1.1  The  large  deviation  principle 

In  practice  one  is  often  interested  in  estimating  the  tail  probability  P  ( /in  (•)  e  A) 
for  certain  sets  A  that  do  not  contain  the  law  of  large  numbers  limit  yu(-).  This 
can  be  studied  in  the  framework  of  a  large  deviation  principle.  We  first  establish  a 
sample  path  large  deviation  principle  for  the  sequence  {/A}ngN.  Asymptotics  of  the 
tail  probabilities  at  a  given  time  t  will  follow  from  the  contraction  principle.  For 
simplicity  we  assume  from  now  on  that  t  G  [0, 1],  while  all  results  in  this  paper  can 
be  established  for  t  in  any  compact  time  interval  by  the  same  argument. 

As  is  explained  in  the  introduction,  a  large  deviation  upper  bound  for  jump- 
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diffusion  Markov  processes  was  obtained  in  [13],  their  local  rate  function  can  be 
identified  with  ours  (see  Section  3.4).  The  more  delicate  part  is  the  proof  of  the 
large  deviation  lower  bound.  Since  the  transition  rates  of  yn  may  tend  to  zero  as 
/in  approaches  the  boundary  of  S,  the  local  rate  function  can  approach  oo,  and  the 
probability  decays  superexponentially.  Our  analysis  requires  that  the  jump  rates  go 
to  zero  at  most  polynomially  when  approaching  the  boundary,  which  is  sufficient  for 
most  of  the  applications,  and  is  formulated  in  the  condition  below. 

Condition  3.1.1.  There  is  a  continuous  function  fo  :  [0, 1]  — >  [0,  oo)  with  fo  (0)  =  0, 
and  constants  C\ ,  C2  <  oo  such  that  for  any  x,y  E  S  and  v  E  V 

d 

(logXi- log yi)  +  f0(\\x  ~y\\)  >  log A„  (x)  -  log A„  (y) 

i= 1 

d 

>  C2^2{logxi-logyi)  -  f0(\\x  -  y\\) . 

i= 1 


Let  int(«S)  denote  the  relative  interior  of  S. 

Remark  3.1.2.  Condition  3.1.1  implies  that  on  any  compact  subset  of  int(«S),  (•) 
is  either  identically  zero  or  uniformly  bounded  below  away  from  zero. 

Another  ingredient  that  is  required  for  the  proof  of  the  large  deviation  lower 
bound  is  an  upper  bound  on  the  time  (or,  a  lower  bound  on  the  speed)  taken  by  the 
law  of  large  numbers  path  to  hit  a  compact  subset  of  int(«S). 

Condition  3.1.3.  There  exist  constants  b  >  0  and  D  <  oo,  such  that  for  any 
x  E  S,  the  law  of  large  numbers  path  y  starting  at  x,  as  defined  by  (2.3.2),  satisfies 
hi  (t)  >  btD  for  t  E  [0, 1] . 

For  t  E  [0, 1]  and  G  C  let  D  ([0,  t]  :  G )  denote  the  space  of  cadlag  functions 
on  [0,  t\  which  takes  value  in  G ,  and  let  AC  ([0,  t\  :  G )  denote  the  space  of  absolutely 
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continuous  functions  on  [0,  t]  that  take  values  in  G.  To  avoid  singularities  in  the  large 
deviation  analysis,  we  need  to  assume  a  certain  communication  condition.  This  is 
formulated  precisely  in  Condition  3.1.4  below.  In  what  follows,  given  t  >  0  and  a 
path  0  G  AC  ([0,  t]  :  5),  let 


Len  ((f>) 


<f)(s) 


ds 


(3.1.1) 


denote  the  length  of  the  image  of  (f>. 

Condition  3.1.4.  There  exist  constants  c  >  0  and  c! ,  p  <  oo,  such  that  for  every 
x  G  S  and  y  Eint(S),  there  exist  t  G  (0, 1),  Fy  G  N,  and  a  piecewise  linear  path  0 
with  (f>  (0)  —  x,  (f)  ( t )  =  y,  such  that 


i).  there  exist  Fx,y  <  Fy,  {t)m}^:,1  C  V,  0  =  t0  <  h  <  ■  ■  ■  <  tFxy  =  t, 
{ Urn } m=i  C  M+  ,  such  that 

Fx,y 

Umvml[trrL_1,trn)  (s)  i  a.e.  s  G  [0,  t] ,  (3.1.2) 

m=  1 


ii).  for  m  =  1, ...,  FX: 


V’ 


A Vm  (0  (s))  >  c  (  min  yA  ,  for  s  G  [tm- 1,  tm), 


Hi).  Len(0)  scales  with  the  Euclidean  distance  between  x  and  y:  Len(0)  < 
c'  \\x-y\\  . 

Definition  3.1.5.  Given  x,y  G  S ,  a  path  0  that  satisfies  properties  i)—iii)  will 
be  said  to  be  a  communicating  path  (with  respect  to  the  constants  c,p,c',  jump 
directions  V  and  jump  rates  {A^j^gyj. 
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Remark  3.1.6.  There  is  some  flexibility  in  the  choice  oft,  {Um}FrfCx  and  { tm }m=i- 
By  a  reparametrization  of  the  path,  we  can  always  assume  t  =  1  and  Um  =  U  for 
every  m.  Moreover,  if  t  is  not  fixed,  we  can  always  choose  Um  =  1.  We  will  make 
these  additional  assumptions  when  convenient. 


We  now  introduce  an  easily  verifiable  sufficient  condition  on  the  jump  rates 
{A„}„gV  which,  together  with  Conditions  3.1.1  and  3.1.4,  implies  Condition  3.1.3. 

Condition  3.1.7.  There  exists  C  <  oo,  such  that  for  any  x  G  S  and  any  v  G  V  and 
i  G  X  that  satisfy  (v,  ef)  <  0,  A„  (x)  <  Cxi. 

Remark  3.1.8.  For  the  interacting  particle  systems  that  we  study,  Xv  (•)  is  specified 
explicitly  by  (2.3.6)  and  (2.3.3),  and  therefore  satisfies  Condition  3.1.7.  Also,  if 
(')Kev  are  Lipschitz  continuous,  then  Condition  3.1.7  can  be  replaced  by:  for 
any  v  G  V  and  i  G  X  that  satisfy  (v,  ef)  <  0,  A„  (x)  =  0  if  Xi  =  0. 

Lemma  3.1.9.  Assume  that  {A„  (-)}„gV  satisfies  Conditions  3.1.1,  3.1.4  and  3.1.7. 
Then  Condition  3.1.3  is  satisfied  with  D  =  J2i=o  for  some  F  <  oo. 


The  proof  of  Lemma  3.1.9  is  deferred  to  Section  3.2.3. 

We  now  state  our  first  large  deviation  result.  Let  Ad_1  =  {x  G  :  Yli=i  xi  =  0}- 
For  x  G  S  and  (3  G  Ad  -1,  define 


L  (x,  (3 )  =  inf  Xv 

Xj2vevvqv=l3^ 


x 


Xv  (x)J  ’ 


(3.1.3) 


where  q  is  a  vector  with  nonnegative  components,  and 


x  log  a;  —  x  +  1  x  >  0, 
oo  x  <  0, 


(3.1.4) 
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is  the  local  rate  function  for  a  standard  Poisson  process.  For  t  G  [0, 1]  and  an 
absolutely  continuous  function  7  :  [0,  t]  1— >  S,  define 

h(l  )=(  L(l(s)  ,7(s))ds,  (3.1.5) 

Jo 

and  in  all  the  other  cases,  set  It  (7)  =  00.  We  will  write  /  (7)  to  denote  I\  (7). 

In  what  follows  we  equip  D  ([0, 1]  :  S )  with  Skorohod  topology,  and  let 
B  (D  ([0, 1]  :  S ))  be  the  associated  Borel  sets. 

Theorem  3.1.10.  Suppose  the  family  of  jump  rates  {A„  (x) ,  x  G  S,v  G  V}  satisfies 
Conditions  2.3.1,  3.1.1,  3.1.3  and  3.1.4 ■  Also,  assume  that  the  initial  conditions 
{pn  (0)}ngN  are  deterministic,  and  fin  (0)  — >  po  £  -P(Af)  as  n  tends  to  infinity. 
Then  the  corresponding  sequence  of  empirical  measure  processes  {p”}neN  satisfies 
the  sample  path  large  deviation  principle  (LDP)  with  rate  function  I .  Specifically, 
for  any  measurable  set  A  G  B  ( D  ([0, 1]  :  S)),  we  have  the  large  deviation  lower  bound 

lim inf  —  log P  (pn  Gl)>  —  inf  {/  (7)  :  7  G  71°,  7(0)  =  /io}  (3.1.6) 

n— xx)  77, 

and  the  upper  bound 

lim  sup  —  log  P  (pn  Gl)<  —inf  {/ (7)  :  7  G  A,  7(0)  =  p0}  .  (3.1.7) 

n— xx)  Tl 

Moreover,  for  any  compact  set  K  C  S  and  M  <  00,  the  set 

{7  G  AC  ([0, 1  \:S):I  (7)  <  M,  7  (0)  G  K}  (3.1.8) 


is  compact. 


The  space  D  ([0, 1]  :  S )  is  naturally  equipped  with  Skorohod  topology.  However, 
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as  is  shown  in  Lemma  2.4  of  [13],  to  prove  the  large  deviation  result  (Theorem 
3.1.10),  one  can  approximate  a  path  in  D  ([0, 1]  :  S)  by  a  piecewise  linear  path,  thus 
it  suffices  to  consider  the  uniform  topology.  The  proof  of  Theorem  3.1.10  is  deferred 
to  Section  3.4  and  3.6.  Applying  the  contraction  principle  (see,  e.g.,  [40])  one  obtains 
the  variational  representation  of  the  rate  function  of  {jin  (t)}ngN  for  any  t  G  [0, 1]. 

Corollary  3.1.11.  Suppose  the  conditions  of  Theorem  3.1.10  hold.  Then  for  each 
t  G  [0,1],  the  sequence  of  random  variables  {gn  (t)}neN  satisfies  the  LDP  with  rate 
function 

Jt  (/i0,  x )  =  inf  {It  (7)  :  7  G  D  ([0, 1]  :  S)  ,  7  (0)  =  g0, 7  (t)  =  x}  (3.1.9) 


3.1.2  A  locally  uniform  refinement 

In  applications,  it  is  often  useful  to  estimate  the  probability  that  yn  hits  a  specific 
point  cn  G  Sn  at  some  given  time,  where  cn  — >  c  G  S  as  n  — >  00.  The  ordinary  LDP 
does  not  imply  an  asymptotic  rate  for  this  hitting  probability  since  it  applies  only 
to  fixed  sets,  and  the  “moving”  set  {cn}  in  the  present  case  has  empty  interior.  To 
obtain  such  a  “locally  uniform”  result  we  need  a  strengthening  of  the  communication 
condition  (Condition  3.1.4),  and  require  local  controllability  of  the  Markov  process 
up  to  the  boundary  of  S.  It  turns  out  that  a  polynomial  lower  bound  for  jump  rates 
up  to  the  boundary  ((3.1.10)  below)  is  sufficient  for  local  controllability. 

Definition  3.1.12.  Given  P,p,p\  <  00  and  c,c  1  >  0,  for  any  x,y  G  S,  a  path 
(f  from  x  to  y  is  said  to  be  strongly  communicating,  if  it  is  communicating  (with 
respect  to  c,  p,  c' ),  and,  in  addition, 

iv).  there  exists  F  <  00,  such  that  supyg5  Fy  <  F , 
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v).  if  4>  has  the  representation  (3.1.2),  then  for  m  =  1  ,...,Fx>y, 


^Vm  (</>(*))  >  Cl 


S  G  [tm_ i,  ^rn)? 


where 


J\fm  { j  ■  f'j  •  V m )  0{  . 


(3.1.10) 


(3.1.11) 


Note  that  when  y  is  on  dS,  min^i ^..tdUi  =  0,  and  therefore  Condition  3.1.4.ii)  is 
trivially  satisfied.  Thus  a  polynomial  lower  bound  (3.1.10)  is  indeed  a  strenghthening 
when  dealing  with  paths  near  the  boundary. 

Definition  3.1.13.  Given  ci  >  0  and  c  ,pi  <  oo,  for  any  x,y  €  Sn,  a  set  of  points 
{0o,  0i, ...,  0A,}  C  Sn  with  0o  =  x,  0fc  =  y  is  called  a  discrete  strongly  communi¬ 
cating  path,  if  they  satisfy  the  following  properties  analogous  to  Definition  3.1.12: 

i).  There  exist  F  <  oo  (independent  of  x  and  y),  C  V,  and  0  =  to  < 

t\  <  ■  ■  ■  <  tp  =  k,  such  that 

0s+l  0s  Xrn ,  S  £  [tm— Ij^to).  (3.1.12) 

n 


ii).  For  all  n  sufficiently  large,  and  m  —  1, ...,  F, 


pi 


Km  (0s)  >  Cl  (  n  (*.),  )  ’  se[tm-i,tri 

\j  G  A/* m 


(3.1.13) 


Hi),  k  <  cn  ||a;  —  y\\. 


In  the  weakly  interacting  particle  systems  we  consider,  {A„  (a;)}  are  specified  by 
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(2.3.6)  and  (2.3.3),  and  each  A„  (x)  is  therefore  a  linear  combination  of  monomials  in 
x  multiplied  by  a  Lipschitz  function  in  x.  Also,  {A”  (x)}  can  be  specified  by  (2.2.6) 
and  (2.2.4).  Thus  the  polynomial  type  lower  bound  in  (3.1.10)  and  (3.1.13)  holds 
for  the  particle  systems. 

Condition  3.1.14.  %).  There  exist  constants  d,F,p,pi  <  oo  and  c,c\  >  0,  such  that 
for  any  x,y  G  S,  there  exists  a  strongly  communicating  path  (f)  that  connects  x  to  y 
(with  respect  to  d ,  F,c,p,c\,pi). 

ii).  For  any  n  G  N  and  any  x,y  G  Sn,  there  exists  a  discrete  strongly  communi¬ 
cating  path  (f)n  that  connects  x  to  y  (with  respect  to  c',  F,  c\,pi). 

Clearly,  Condition  3.1.14  is  a  strengthening  of  Condition  3.1.4.  We  now  state  the 
locally  uniform  LDP  result,  which  is  proved  in  Section  3.7. 

Theorem  3.1.15.  Suppose  the  family  of  jump  rates  {A”  (x)  ,x  G  S.v  G  V}ngN  sat¬ 
isfies  Conditions  2.3.1,  3.1.1,  3.1.3  and  3.1.  If .  Also,  assume  the  initial  conditions 
{pn  (0)}ngN  are  deterministic,  and  pn  (0)  — >  po  G  P  (X)  as  n  tends  to  infinity.  Let 
{^n}neN  C  Sn,  x  G  S  be  such  that  xn  — >  x  as  n  — >  oo.  Then  for  any  t  G  [0, 1), 

lim  -  logP  (nn  (■ t )  =  xn)  =  -Jt  (po,  x) , 

n— xx)  fi 


where  Jt  is  as  defined  in  (3.1.9). 


3.1.3  LDP  for  invariant  measures 

In  [20],  a  uniform  sample  path  LDP  for  small  noise  diffusions  (with  respect  to  initial 
conditions)  is  used  to  study  its  metastability  properties,  including  the  mean  exit 
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time,  most  likely  exit  location  from  a  given  domain,  and  the  LDP  for  invariant 
measures.  The  program  of  [20]  was  carried  out  for  non-degenerate  diffusions  in 
here  we  have  a  sequence  of  jump  processes  on  lattice  approximations  of  a  compact  set. 
However,  we  remark  here  that  when  Condition  3.1.14.i)  holds,  the  same  arguments 
carry  through  without  essential  change.  In  [20]  extra  conditions  are  assumed  to 
guarantee  that  the  process  does  not  escape  to  infinity  with  significant  probability; 
for  our  model,  since  the  state  space  is  compact,  this  is  automatic.  The  quasipotential 
is  defined  by 

V  ( x ,  y)  =  inf  {It  (7)  :  7  e  D  ([0,  t]  :  S) ,  7  (0)  =  x,  7  (t)  =  y,  t  <  00}  ,  for  x,y  eS. 

(3.1.14) 

We  need  the  quasipotential  to  be  continuous  within  its  domain.  This  property  follows 
from  the  fact  that  for  any  x,y  E  S  that  are  sufficiently  close,  one  can  construct  a 
path  connects  x  to  y  with  arbitrary  small  cost  (Lemma  3.7.1). 

Lemma  3.1.16.  Assume  Condition  3.1.1  j.i)  holds,  then  V  (•,  •)  is  jointly  continuous 

in  S  x  S. 

The  proof  of  Lemma  3.1.16  is  deferred  to  Section  3.7.  We  now  state  the  LDP  for 
invariant  measures  in  the  case  of  a  single  equilibrium. 

Theorem  3.1.17.  Assume  that  xq  is  the  unique  stable  equilibrium  of  the  LLN  dy¬ 
namics  (2.3.2),  and  is  globally  attracting  (in  S).  Also  assume  Condition  S.l.lf.i) 
holds.  Then  for  any  n  >  0,  there  exists  a  unique  invariant  measure  vn  of  the  Markov 
process  with  generator  (2.2.5).  Moreover,  {un}neN  satisfies  an  LDP  with  rate  func¬ 
tion  V  (xo,  •). 

When  the  LLN  limit  (2.3.2)  has  multiple  stable  equilibria,  following  the  same 
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approach  as  the  case  of  non-degenerate  diffusions  (see  [20],  Chapter  6.4),  a  general¬ 
ization  of  the  above  theorem  also  holds. 


3.1.4  Corollaries  for  the  mean  field  interacting  particle  sys¬ 
tems 


We  now  describe  how  the  results  in  Sections  3.1.1  and  3.1.2  apply  to  the  particular 
setting  of  the  mean  held  interacting  particle  systems  introduced  in  Chapter  2. 


Notice  that  in  the  statements  of  Theorems  3.1.10  and  3.1.15,  one  can  rewrite 
the  local  rate  function  (3.1.3)  as  a  function  of  the  jump  rates  (a-)}  (see  (2.3.3)) 
of  the  particle  systems.  For  this,  we  need  to  introduce  a  natural  ordering  for 
pairs  (i,j)  G  U k<KJk,  by  first  ordering  them  by  increasing  k,  and  then  ordering 
((*i, ..,  ifc) ,  (ji, ...,  jk))  lexicographically.  Denote  by  Ck  the  cardinality  of  U k<K<Jk- 
This  ordering  establishes  a  bijection  between  natural  numbers  (1  ,...,Ck}  and  or¬ 
dered  pairs  (i,  j)  G  C\k<xJk  ■  With  this,  we  sometimes  abuse  the  notation  and  refer 
to  r  G  N  instead  of  r  —  e j  —  ei  —  Ym=i  ek  ~  Y^i= i  eir 


Let  W  be  a  d  x  Ck  matrix  with  each  column  specified  as  the  corresponding  jump 
direction:  Wr  —  e-3  —  e\  if  r  G  N  maps  to  ej  —  e\.  For  x  G  S  and  (3  G  Ad_1,  (3.1.3) 
can  be  written  as 


L{x,/3) 


K 


q- 


inf 

Wq=/3 


£ 


(ij)eyfc 


where  q  =  diagjy1,  ,..,qK)  is  a  block  matrix  such  that  each  qk  is  a  dk  x  dk  matrix 
with  nonnegative  entries  for  (i, j)  G  Jk ,  and  zero  for  (i, j)  G  X2k\Jk .  In  the  special 
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case  that  K  —  1,  L  can  be  written  as 


L(x,(3) 


d  d 

inf  E  E 


atj  [x) 


5 


and  the  constraint  W q  =  (3  reduces  to 


E  ~  E  qF  =  J  =  •••’  d' 

where  we  represent  q 1  simply  by  q. 


Conditions  2.3.1,  3.1.1  and  3.1.7  are  properties  of  {A„  (a;)},  and  therefore  can  be 
directly  verified  from  (2.3.6).  Conditions  3.1.4  and  3.1.14  hold  for  many  classes  of 
interacting  particle  systems  with  simultaneous  jumps.  Here,  we  state  three  easily 
verifiable  sufficient  conditions  in  terms  of  the  T  matrices  of  the  original  particle 
systems.  For  k  <  K  and  (i,j)  G  Jk ,  denote 


(3.1.15) 


and  let 

Uk=  {(i,j)  G  Jk  :  2^  >  0}  . 

The  first  condition  states  that  an  entry  of  the  rate  matrices  in  a  given  jump  direction 
is  either  identically  zero,  or  uniformly  bounded  below  away  from  zero  (note  that  the 
jump  rate  cty  in  that  direction  is  not  bounded  away  from  zero). 

Condition  3.1.18.  Fork  =  1,  ...,K  and  (i,j)  G  Jk ,  either  (i,j)  G  J\fk ,  or  Th  (x)  =  0 
for  every  x  G  S . 

Condition  3.1.19.  For  every  x  G  S,  the  Markov  process  on  X  with  rate  matrix 
T1  (x)  is  ergodic. 


44 

30 

Condition  3.1.20.  For  every  x  G  S,  the  Markov  process  on  X  with  rate  matrix 
rcff  (x)  is  ergodic,  where  refr  is  defined  as  in  (2.3.5). 

In  Section  3.2,  we  establish  the  following  result. 

Proposition  3.1.21.  The  following  two  assertions  hold: 

i) .  Conditions  3.1.18  and  3.1.20  imply  Condition  3.1.4- 

ii) .  Condition  3.1.19  and  the  continuity  of  V1  (•)  implies  Condition  3.1.14  (and 
hence,  also  Condition  3.1-4)- 

In  particular,  both  the  LDP  and  the  locally  uniform  LDP  hold  for  the  interacting 
particle  systems  described  in  Examples  2.4-1,  2-4-2  and  2-4-3. 

For  the  n— particle  systems  we  study,  it  is  more  natural  to  start  with  random 
initial  conditions.  Depending  on  the  large  deviation  rate  of  the  initial  condition,  this 
gives  rise  to  an  additional  cost  in  the  rate  function.  The  LDP  for  empirical  measure 
processes  with  random  initial  conditions  are  stated  in  the  following  corollary. 

Corollary  3.1.22.  Assume  Conditions  2.3.1,  3.1.1,  3.1.3,  3.1.18  and  3.1.20  hold. 
Also,  assume  that  the  empirical  measure  processes  {/in}ngN  are  Markovian,  with 
generator  (2.2.3)  and  initial  conditions  {/in(0)}ngN  that  converges  to  po,  in  such 
a  way  that  they  satisfy  an  LDP  with  rate  function  Jo  (•).  Then  the  corresponding 
sequence  of  empirical  measure  processes  {/in}ngN  satisfies  the  sample  path  LDP  with 
rate  function  Jo  (7  (0))  +  I  (7). 

Proof.  For  any  y  G  Sn,  denote  Un  ( y )  =  — ^logE ye~nh^C) ,  By  Theorem  3.1.10  and 
the  equivalence  between  the  LDP  and  the  Laplace  principle  [40] ,  for  any  sequence  of 
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{yn}nGf  J  that  converges  to  some  y  G  S, 

Un  (yn)  -»• U(y)  =  inf  {/  (7)  +  h  (7)  :  7  G  D  ([0, 1]  :  S) ,  7  (0)  =  y}  . 

By  Lemma  3.1.23  below,  U  is  continuous  on  S.  A  standard  argument  by  contradic¬ 
tion  then  shows  that  Un  converges  to  U  uniformly.  Let  un  denote  the  law  of  /jn  (0). 
Then 


lim  — —  log  EMn(0)e  = 

n^oo  Tl 


where  the  third  equality  follows  from  the  assumption  and  the  continuity  of  U.  The 
conclusion  follows  by  the  equivalence  between  the  LDP  and  the  Laplace  principle.  □ 

Lemma  3.1.23.  U  (y)  =  inf  {/  (7)  +  h  (7)  :  7  e  D  ([0, 1]  :  S) ,  7  (0)  =  y}  is  continu¬ 
ous. 

Proof.  Fix  e  >  0.  Given  5  >  0  such  that  c(S)  <  e/3,  and  any  ys  e  S  such  that 
\\ys  —  y\\  <  5,  by  Lemma  3.7.1,  there  exists  a  path  v  e  AC  ([0,  5]  :  S)  with  v  (0)  =  y , 
v  (5)  =  ys  such  that  Is  iy)  <  c(S).  And,  by  the  construction  in  the  proof  of  Lemma 
3.7.1,  supser0  (5i  ||i/(s)  —y ||  <  C8  for  some  C  <  00.  We  now  rescale  7  to  obtain  a 
new  path  7*5:  for  c  =  (1  —  5)_1,  define  75  G  AC  ([0, 1  —  5]  :  5)  by  y*5  (s)  =  7  (cs).  By 
Proposition  3.5.8,  we  can  take  8  smaller  if  necessary  such  that  I\s  (y*5)  <  /  (y)+e/3. 
Let  7  be  the  concatenation  of  v  and  y*5,  ||y  —  yH^  — >  0  as  8  — >  0.  Therefore 

U  (y5)  <  h( 7)  +  I  (7)  <  I  (7)  +  e/3  +  c  (5)  +  h  (7)  -  h  (7) , 


lim  ——log  e  {//„  }• 

n—>oo  77,  ^  ^ 

y-n^Sn 

lim  --log  [  e-n{u{y)+oW)un  (dy) 

n—>  00  77,  J 

inf  {U  (y)  +  J0(y)} 

y£S 

inf  {h  (7)  +  J0  (7  (0))  +  I  (7)}  , 

7eD([0,l]:5) 


46 


32 


which  goes  to  zero  by  sending  <5^0  and  then  sending  e  — >  0.  The  other  inequality 
is  proved  in  the  same  way.  □ 

Example  3.1.24.  Assume  that  in  the  n— particle  system,  particles  are  initially  dis¬ 
tributed  as  i.i.d  X—  valued  random  variables,  with  common  distribution  v.  Then  by 
Sanov’s  theorem,  Jo  (fJ>o)  =  R  (/x0  \\u)  =  ffdi=1  ho log 


The  analogous  result  holds  for  the  locally  uniform  refinement  of  the  LDP. 


We  note  that  these  are  only  sufficient  conditions,  and  the  LDP  can  be  shown 
to  hold  for  other  interacting  particle  systems,  by  directly  verifying  the  conditions  of 
Theorem  3.1.10  (especially,  Condition  3.1.4),  as  illustrated  by  the  following  example. 

Example  3.1.25.  Let  d  —  4,  K  =  2,  and  define  the  generator  of  the  Markov  process 
{//*}  as  in  (2.2.3)  with  a\jn  defined  as  in  (2.2.4),  terms  ofT1,n  and  T2,n  given  by 


ri2  0*0  =  Cl,  Llfi  ( X )  =  c2, 

^  0*0  =  C4,  r(G),(3,4)  (X)  =  C5> 


t~\2,72 

(3, 4), (1,2) 


(x)  =  C6 


for  Ci  >  0,  i  =  1,...,6,  for  all  x  E  S,  and  identically  zero  otherwise.  The  effective 
jump  rate  matrix  rcfr  then  takes  the  form 


r!l  0*0  =  ci,  r|f  (x)  =  c2,  (x)  =  c3,  rif  (x)  =  c4 

Ti!  (x)  =  X2c5,  r|f  (x)  =  x4c6,  r24  (x)  =  X! c5,  Tf2  (x)  =  x3c6. 

rcff  is  not  ergodic  on  the  part  of  the  boundary  given  by 

{x  G  S  :  x3  =  x4  =  0  or  x\  =  x2  =  0}.  However,  one  can  easily  verify  Condition 

3.1.4,  as  well  as  Conditions  2.3.1,  3.1.1  and  3.1.7. 

Given  a  mean  field  interacting  particle  system  with  simultaneous  jumps,  one  can 
construct  another  single  jump  particle  system,  according  to  the  effective  jump  rate 
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matrix  {rn,eff  j  defined  as  in  (2.3.4).  As  mentioned  in  Chapter  2,  these  systems  have 
the  same  LLN  limit.  However,  the  following  example  shows  that  the  prelimit  models, 
in  particular  their  large  deviation  behavior,  can  have  a  significantly  different  nature. 

Example  3.1.26.  Let  X  =  {1,  2,  3, 4,  5,  6},  K  =  2,  and  for  any  n  e  N  consider  a 
Markov  process  with  the  following  properties:  states  are  grouped  into  blocks  {1,2}, 
{3,4},  {5,6},  such  that  within  each  block,  states  communicate  through  single  jumps, 
and  there  exist  simultaneous  jumps  of  two  particles  between  different  blocks.  Also, 
the  number  of  particles  in  the  first  block  can  only  change  by  an  even  number  during 
each  jump.  In  particular,  consider  the  Markov  process  associated  with  the  generator 
(2.2.3),  with  akn  defined  as  in  (2.2.4),  terms  of  T1,n  and  Y2,n  given  by 


■pl,H 
1  12 

(x) 

=  i,r 

=  (*) 

=  i,  hirer; 

)  =  1,  r5fT 

(x)  =  1,  Tgf  (x 

i2,n 

(1.2), (3, 4) 

(x) 

=  1, 

h(3, 4), (1,2)  (X)  =  1’ 

r2,n  i 

1  (2, 3), (4,1)  1 

(x)  =  1, 

rHb,(2,3)  (*)  = 

i2,n 

(1,3), (4, 2) 

(x) 

=  1, 

^(1,3), (5, 2)  ( X )  =  1’ 

r2,n  i 

1  (1,4), (5, 2)  1 

(®)  =  !, 

h(i,6),(5,2)  ( X )  = 

\2  ,n 

(2, 4), (1,5) 

(x) 

=  1, 

h(2, 5), (1,4)  ( X )  = 

r2,n  i 

1  (1,5), (6, 2) 

(®)  =  !, 

h(3,5),(6,l)  (X)  = 

\2  ,n 

(4, 6), (1,2) 

(x) 

=  1, 

h(l,2),(5,6)  ( X )  =  1’ 

r2,n  i 

1  (5, 6), (1,2)  1 

(®)  =  !, 

h(5,6),(3,4)  ( X )  = 

for  all  x  E  S,  and  identically  zero  otherwise.  One  computes  the  effective  jump  rate 
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matrix  to  be 


peff 

1  12 

(x) 

=  1, 

peff 
1  21 

0*0 

=  1  +  x4  +  x5, 

peff 
1  34 

0*0 

=  1, 

■peff 

1  43 

0*0 

=  1, 

peff 
1  56 

0*0 

=  1, 

peff 
1  65 

0*0 

=  1, 

peff 

1  13 

(x) 

=  x2  +  x4, 

peff 
1  24 

0*0 

=  X\  +  x3, 

peff 
1  31 

0*0 

=  x2  +  x4 

peff 
1  42 

0*0 

=  2xi  +  x3, 

peff 
1  14 

0*0 

=  X3, 

peff 
1  32 

0*0 

=  2xu 

peff 

1  15 

0*0 

=  X2  +  x3  +  XA  +  x6, 

peff 
1  62 

0*0 

=  Xi  +  x4  +  x5, 

peff 
1  45 

0*0 

=  X2, 

peff 

1  54 

0*0 

=  X2, 

peff 
1  16 

0*0 

=  X5, 

peff 
1  52 

0*0 

=  Xi, 

peff 
1  36 

0*0 

=  X5, 

peff 
1  51 

0*0 

=  X3  +  x6, 

peff 
1  41 

0*0 

=  xe, 

peff 

1  53 

0*0 

=  X6, 

peff 
1  64 

0*0 

=  X5, 

peff 
1  26 

0*0 

=  Xi. 

Conditions  2.3.1,  3.1.1,  3.1.3  and  3.1.18  can  be  checked  explicitly  from  (2.3.3)  and 
(2.3.6).  Also,  Condition  3.1.20  follows  from  the  fact  that  reff  (x)  is  affine  in  x,  and 
is  ergodic  for  i  G  {1, 6}.  Thus,  the  associated  single  jump  particle 
system  satisfies  an  LDP  and  a  locally  uniform  LDP  by  Theorem  3.1.10  and  3.1.15. 
In  particular,  for  any  x,y  G  S,  if  we  take  xn,yn  G  Sn  such  that  ||a:n  —  a;||  — >  0, 
\\yn  —  y\\  — ►  0,  Theorem  3.1.15  implies 

(on  (t)  =  yn)  =  exp  (- nJt  (x,  y)  +  o  (n)) ,  as  n  ->  oo, 

where  Jt  (•)  is  defined  as  in  (3.1.9)  and  is  finite  because  of  the  communication  prop¬ 
erty.  However,  the  original  simultaneous  jump  particle  system  fails  to  satisfy  the 
locally  uniform  LDP.  Indeed,  for  any  n  G  N,  the  number  of  particles  in  states  {1,2} 
remains  either  odd  or  even.  Therefore  if  we  take  xn,yn  G  Sn  that  additionally  satisfy 
{xf  +  x7) }neN  is  even  mid  {y7(  +  y% }neN  is  odd,  then  Px„  (n11  (t)  =  yn)  =  0  for  any 
n  G  N. 
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3.2  Verification  of  Conditions 


In  this  section  we  verify  that  the  assumptions  of  Theorems  3.1.10  and  3.1.15  are 
satisfied  by  a  large  class  of  mean  field  interacting  particle  systems.  Readers  only 
interested  in  the  large  deviation  proof  can  skip  this  section. 

Recall  the  definition  of  QTty  in  (3.1.15),  and  let 

c0  =  min  :  97ly  >  0,  k  =  1, K,  (i,  j)  G  Jk)  >  0. 

3.2.1  Proof  of  Proposition  3.1.21.i) 

To  prove  Condition  3.1.4  under  Conditions  3.1.18  and  3.1.20,  we  will  actually  prove 
a  stronger  result.  Namely,  we  show  that  Condition  3.1.4  holds  under  a  much  weaker 
communication  condition  for  the  Markov  processes  on  X ,  introduced  below  as  Defi¬ 
nition  3.2.1. 

Definition  3.2.1.  For  two  states  u,  v  E  X ,  v  is  said  to  be  K  —  accessible  from  u,  if 
there  exist  L  €  {1,  and  a  sequence  of  distinct  states  in  X:  u  =  ui,U2, ...,  ul  =  v, 
such  that  for  m  —  1, ...,  L  —  1, 

i) .  there  exist  krn  e  {1, ...,  K },  (im,  jm)  e  Jkm ,  and  lm ,  l'm  £  {1, ...,  km},  such  that 

a rn  iin,lm  and  um- )_1  3m,l'm’ 

ii)  •  fori  1 .... .  A',,, .  im,i  £  { ^  i  ■  •  •  •  -a  m  \  ■ 

in).  Tlfm.  >  0. 

7  imjm 
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If  for  any  u,  v  G  X ,  v  is  K — accessible  from  u,  we  call  the  Markov  process  asso¬ 
ciated  with  {r y  (-)}(i  i)euk<Kjk  K—er9°dic. 

Notice  that  when  K  =  1,  the  notion  of  1— ergodicity  coincides  with  the  ergod- 
icity  of  single  jump  rate  matrix  T1.  Roughly  speaking,  the  general  K— accessibility 
condition  ensures  that  one  can  reach  state  v  from  u,  by  going  through  certain  types 
of  simultaneous  jumps,  such  that  for  each  m,  one  can  move  mass  exclusively  from  a 
subset  of  states  {ui,  to  um+ i  (this  is  ensured  by  the  second  condition  of  Defi¬ 

nition  3.2.1).  Up  to  our  knowledge,  all  the  interesting  examples  satisfy  the  conditions 
of  Definition  3.2.1.  And,  the  second  condition  allows  one  to  construct  communication 
paths,  by  moving  along  the  direction  of  such  sequences  of  simultaneous  jumps. 

To  illustrate  the  notion  of  K— ergodicity,  note  that  the  interacting  particle  system 
with  simultaneous  jumps  described  in  Examples  3.1.25  and  3.1.26  are  2— ergodic.  Let 
us  verify  2— ergodicity  for  Example  3.1.25.  Take  any  i,j  G  {1,2,  3, 4},  and  assume 
without  loss  of  generality  i  —  1,  by  the  symmetry  of  the  dynamics.  If  j  =  2,  we  can 
take  L  —  2,  k\  —  1  and  (ii,  ji)  =  (1,  2).  If  j  =  3,  take  L  —  3,  u\  —  1,  U2  —  2,  w3  =  3, 
fci  =  1,  (ii,  ji)  =  (1,  2),  /c2  =  2  and  (i2,  j2)  =  ((1,  2) ,  (3, 4)).  The  j  =  4  case  is  similar 
to  j  =  3,  except  that  w3  =  4.  It  is  easy  to  check  the  sequence  of  states  { um }  and 
jumps  satisfy  conditions  i),  ii),  iii)  of  Definition  3.2.1. 

Remark  3.2.2.  We  show  that  Conditions  3.1.18  and  3.1.20  imply  K—  ergodicity  of 
{Pij  (^)}(ij )(zuk<Kjk>  for  any  x  ^  S-  Take  any  u,v  G  X,  u  ^  v.  Since  Tcff  (eu)  is 
ergodic  by  Condition  3.1.20,  there  exists  a  sequence  of  distinct  states  u  =  u±,  ...,ul  = 
v  such  that  rfliUrn  1  ( eu )  >  0 ,  m  —  1  ,...,L  —  1.  By  the  definition  of  Teff  given  in 
(2.3.5),  this  implies  that  for  m  =  1,  ...,L  —  1,  there  exist  krn  G  {1, ...,  K},  (im,jTO)  G 
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«. 7  m?  CLTld  IfYi  G  •••)  SUCH  that  Urn  ^ra,Zm?  ^m+1  jm,lm 

km 

n  (eu,  eim,r)  r;  ™jm  (eU)  >  0. 

r=l 

7*7 ^lm 

This  implies  that  3JI?’’ >  0,  and  im>r  =  u  for  every  r  lm .  hi  other  words, 
the  l %  component  of  \m  is  equal  to  um,  and  all  other  components  are  equal  to  u. 
Definition  3.2. l.i)  is  satisfied  with  lm  =  l'm,  and  Definition  3.2.1.H)  is  satisfied  with 
im,i  G  {u,  um}  fori  =  1  Finally,  since  (eu)  >  0,  Condition  3.1.18 

implies  that  >  0,  property  Hi)  of  Definition  3.2.1  is  also  satisfied. 

To  prove  Condition  3.1.4,  one  needs  to  show  some  local  controllability  property 
of  the  underlying  Markov  process.  More  precisely,  it  suffices  to  show  that  there  exist 
constants  c  >  0  and  c',p  <  oo  such  that  for  every  x  G  S  and  y  G  int(«S),  there 
exists  a  communicating  path  0  from  x  to  y  (with  respect  to  the  constants  c,p  and 
cr),  as  stated  in  Definition  3.1.4.  We  show  that  K— ergodicity  provides  sufficient 
controllability.  First,  it  allows  one  to  move  from  any  point  on  the  boundary  to  some 
compact  convex  subset  of  int(«S)  that  contains  y ,  using  a  sequence  of  jumps  whose 
rates  are  uniformly  bounded  below  away  from  zero  (Lemma  3.2.3).  Then,  we  show 
that  in  the  compact  subset,  the  controllability  is  stronger:  one  can  in  fact  move 
toward  any  coordinate  direction  (Lemma  3.2.4),  and  thus,  the  construction  of  such 
a  path  in  this  compact  subset  is  straightforward  (Lemma  3.2.6). 

For  a  >  0  define  Sa=  {x  e  S  :  Xi  >  a,  i  —  1, ...,  d}. 

Lemma  3.2.3.  Assume  that  for  anyi,j  e  X ,  j  is  K—  accessible  from  i.  Then  for  any 
x  G  S  and  a  G  [0, 1/  (. K  +  l)d_1  d],  there  exist  z  G  Sa ,  t0  >  0  and  a  communicating 
path  0  G  C  ([0,  to]  :  S)  that  connects  x  to  z.  Moreover, 
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i) .  there  is  C  <  oo  such  that  for  any  x  G  S,  0  can  be  chosen  such  that 
fo°  110  (s)  IMS  <  Cdist(x,Sa), 

ii) .  for  any  s  e  [0,  to]  and  i  G  X  such  that  fa  (s)  <  0,  fa  (s)  >  a. 

Before  proving  the  lemma  in  general  let  us  first  illustrate  with  Example  3.1.25 
where  d  —  4  and  K  =  2.  Assume  without  loss  of  generality  x\  >  X2  >  x3  >  x4,  and 
therefore  X\  >  1/4.  As  before,  we  take  (ii , )  =  (1,2),  and  (12J2)  =  ((1,2) ,  (3,4)). 
For  a  <  1/108,  we  discuss  the  construction  of  such  a  path  in  three  cases  that  exhaust 
all  possibilities. 

Case  I.  x3  >  a  >  x4.  Take  0  (0)  =x,<f>  (t)  =  (eh  -  eh)  l[0i2(o-x4))  (0 
+  (ej2  —  eh )  l[2(a-x4),3(a-x4))  (t) .  It  is  easy  to  verify  property  i)  in  the  lemma.  To  verify 
property  ii),  note  that  for  t  G  [0,  3  (a  —  £4)],  fa  (t)  —  0i  (0)  —  2  (a  —  x4)  —  (a  —  x4)  > 
xi  —  3a  >  a,  02  (t)  >  02  (0)  >  a,  fa  (t)  >  0,  fa  (t)  >  0.  Since  04  (3  (a  —  x4))  =  a,  we 
have  0  (3  (a  —  x4))  G  cSa. 

Case  II.  x2  >  a  >  x3.  Take  0(1)  (0)  =  x,  0(1)  (f)  =  (e^  -  eh)  l[0,2(o-x3))  (t)  + 
(ej2  —  eb)  l[2(a— x3),3(a— x3))  (t)-  One  can  verify  property  i)  and  ii)  in  the  lemma  are 
satisfied  for  0T)  by  the  same  argument  applied  to  0.  If  =  fa1^  (3  (a  —  x3)),  then 
we  have  >  a,  ztf*  >  a,  Zg1'1  >  a.  Then  it  is  reduced  to  Case  I,  and  we  can  further 
take  a  communicating  path  that  moves  z ^  into  Sa,  and  concatenates  with  0T)  to 
obtain  the  desired  path. 

Case  III.  x\  >  a  >  x2.  Take  fa2)  (0)  =  x,  fa2^  (t)  =  (ej1  —  efa  l[0,o-x2)  (t)-  One 
can  verify  property  i)  and  ii)  in  the  lemma  are  satisfied  for  fa2\  And  at  t  =  a  —  x2, 
xi  >  a  =  x2  >  x3  >  X4,  and  it  reduces  to  Case  II. 
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The  construction  of  a  communication  path  can  be  generalized  into  the  following 
proof. 

Proof.  If  a  =  0  or  x  G  Sa,  we  can  choose  z  =  x  and  there  is  nothing  to  prove. 
Therefore  we  assume  a  >  0  and  x  ^  Sa.  Assume  without  loss  of  generality  that 
x\  >  X2  >  •  •  •  >  Xd,  and  for  some  2  <  k  <  d,  Xk-i  >  a  >  Xk-  When  K  =  1,  a  more 
straightforward  proof  (which  gives  stronger  result)  is  given  in  Lemma  3.2.7  below. 
For  general  K,  we  will  prove  the  lemma  by  induction  on  w  ==  d  —  k,  one  less  than 
the  number  of  indicies  smaller  than  a. 

If  w  —  0,  then  Xd  is  the  only  component  such  that  Xd  <  a.  By  the  definition 
of  K— accessibility  there  exist  L  G  {l,...,d}  and  a  sequence  of  distinct  states  1  = 
Mi,  U2,  ul  =  d,  such  that  form  =  —  1,  there  exist  krn  e  {1  (im,jm)  G 

Jkm,  and  lm ,  l'm  G  {1, ...,  km},  such  that  um  =  um+i  =  jm/m,  and  >  0. 

Now  let 

|  (K  +  1)L~2  -  (K  +  if-1-™  if  m  G  {1, ...,  L  —  1}  , 

(  (K  +  l)L~2  if  m  =  L. 

Then  consider  the  piecewise  linear  path  f  with  <f  (0)  =  x,  and 

L—l 

&  W  =  'y  ^  (ej m  ~  ei m)  Mcm,L(.a-xd),cm+1'L{a-xd))  (t)  J  a.e.  t  >  0.  (3.2.1) 

771=  1 

Let  to  =  (K  +  1)L  2  (a  —  Xd).  We  claim  that  for  l  =  2,  ...,d  —  1,  and  t*  G  [0,  to] 
such  that  flit*)  <  0,  4>i(s)  >  a  for  s  G  [t*,t0].  Indeed,  for  such  /,  there  exist 
m  G  —  1}  and  r  G  {1  ,...,km}  such  that  l  =  im,r.  If  there  exist  multiple 

(m,  r)  that  satisfy  l  =  im,r,  we  take  the  pair  with  smallest  m.  Then  by  Definition 
3.2. l.ii) ,  l  G  {«i, ...,  thus  there  exist  m!  G  —  1}  and  r’  G  {1, ...,  km>} 
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such  that  /  =  jm>y.  Therefore,  for  any  s  e  [t*,t0] , 

4>i  0)  =  4n  (o)  +  f  <j>i  (t)  dt 

J  o 

^  O  T  ^Cj m, ,  (Cm'+l,i  Cm' ,l)  (fl  *^d) 

L—l 

^  ^  (d-p+l,L  CPiL)  (a  S'd) 

p=m 

>  a, 


where  the  last  inequality  follows  from  the  fact  that  (e\ pie/)  <  Jl ,  and 

L— 1 

Cm'+1,L  Cm',L  ^  Cm,L  Cm— 1,L  A  A  ^  ^  (cp+l,L  ^P,l)  ■ 

p=m 

Also,  as  will  explained  below,  for  s  e  [0,  to], 


0)  =  (0)  +  /  0i  (t)  dt 


L- 1 


>  ^  (eip-ei>  (cP+i,i  -  cP,i)  (°  -  ®d) 


p=i 


KCL’L  (K  +  l)d_1  d 

1  1 
>  - 


d  (K  +  1)  d 
>  a. 


Here  the  first  inequality  follows  since  Y^i= i  xi  —  1  and  >  •  •  •  >  Xd  implies  x\  > 
1/d.  The  second  inequality  follows  from  (eipiei)  <  A'  and  a  —  Xd  <  a  <  ,lf/. 

The  third  inequality  follows  since  cl,l  =  ( K  +  1)L  2  <  (Jl  +  l)cZ~2,  and  the  fourth 


since  K/K  +  1  >  1/(K  +  l)d  1. 
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Finally,  by  Definition  3.2. 1  .ii) ,  <fid(s)  >  0,  and  since  (e-}L1  —  e\ L_1,ed)  >  1, 

fto 

(pd  (Jo)  =  xd+  (fid  fit)  dt 
Jo 

(■CL,L{a-Xd) 

>xd+  (fid  fit)  dt 

j  CL-i,L(a-xd) 

A  Xd  +  erf)  ( Cl  l  cl~i,l)  (((  *£d) 

>  xd  +  a  -  xd  —  a. 

Therefore  (fi  (to)  G  <Sa,  and  property  ii)  of  Lemma  3.2.3  is  satisfied.  Also,  0  is  a 
communicating  path,  since  by  (2.3.3)  and  (2.3.6), 


a  >«£:,„  ww) 


(km)\  P 

. 


If*».-Wrt"j„(^W) 


Property  i)  is  satisfied  since  fQ°  \\<fi  ( s )  ||ds  <  \Z2Kt0  <  C2  ( d ,  K )  (a  —  xd) 
<  C'2  ( d ,  iP)dist(x,  Sa)  for  some  C2  <  00. 


Suppose  the  conclusion  of  the  lemma  holds  for  w  <  h,  and  now  let  w  =  h  +  1 
(and  k  =  d  —  h  —  1).  The  idea  is  first  move  along  a  communicating  path  that 
uses  the  same  jump  directions  as  (3.2.1),  until  reaching  some  z^l\  such  that  for 
some  s  G  {k,...,d},  zP  >  a,  and  then  use  the  induction  assumption.  To  con¬ 
struct  such  a  path  (fP-\  take  the  same  sequence  of  jumps  {(im,jm)}  as  in  the  case 
of  w  =  0.  Let  mo  be  the  smallest  index  such  that  one  of  the  states  with  value 
below  a  can  be  increased:  mo  =  min{m  :  3/  s.t.  3s  G  {k, ...,  d}  ,  s  Let 
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A  =  min  {a  —  xs  :  s  G  {k, d}  ,  s.t.  3/,  s  =  We  then  define 


(. K  +  l)m°-2  -  (AT  +  1) 


mo-l-m 


-m,ra  o 


(/i  +  1) 


mo— 2 


m  G  {1,  -  1}  , 

m  =  mo, 


and  let  0^  be  the  path  given  by  (0)  =  x, 


m0-l 


0(1)  (*)  =  (eJ-  _  ei-)  ilcm.moA^+i^oA)  (t) ,  a.e.  t  >  0. 


m=  1 


Also,  let  =  (j)^  (cmo,moA).  Then  by  the  same  argument  applied  to  the  path  0,  0(^ 
is  a  communicating  path,  and  for  t*  G  [0,  cTOQimoA]  and  /  €  X  such  that  <p'lX)  (t*)  < 
0,  it  follows  that  0^  (s)  >  a  for  s  G  [t*,cm0)mo A],  Also,  J0Cm°’m°A  H^1)  (s)  ||ds  < 
V2Kcmo>rnoA  <  C‘2  (i d ,  iL)dist(x,  «S°).  Therefore,  for  l  G  {1, ...,  k  —  1},  >  a,  and 

there  exists  s  G  {/c, ...,  d},  such  that  >  a.  Thus  at  most  h  components  of  are 
less  than  a.  By  the  induction  assumption  there  exists  z  G  Sa,  and  a  communicating 
path  that  connects  z ^  to  z,  and  satisfies  properties  i)  and  ii).  The  conclusion 
then  follows  by  concatenating  0^  and  <f>^2\  and  noticing  that 


dist  (z(1),<Sa)  <  ||x  —  ^(1)  ||  +  dist  (x,  Sa^ 

/Cm0,m0A  , 

||0  ^  (s)  || ds  +  dist  Sa 
<  (C2  (d,  K)  +  1)  dist  (x,  . 


□ 

Lemma  3.2.4.  Assume  that  for  any  i,j  G  X ,  j  is  K—  accessible  from  i.  Then  for 
any  u,  v  G  X,  u  ^  v,  there  exist  a  finite  constant  F  =  FUjV,  km  G  {!,...,  K}  for 
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m  =  am  >  0  and  (im,jm)  G  Jkm  such  that  >  0,  such  that 


F 


771=1 


(3.2.2) 


We  first  illustrate  with  Example  3.1.25.  Without  loss  of  generality,  we  set  v  —  4 

and  u  —  1.  As  before,  we  take  (iiJ-J  =  (1,2),  and  (12J2)  =  ((1,  2) ,  (3, 4)).  Thus 

(eji  “  eu)  +  (ej2  -  ei2)  =  e3  +  e4  -  2ei. 

To  cancel  the  term  e3,  we  further  take  (i3,j3)  =  (3,4).  Then  J2m=i  ( ej m  ~~  Dm)  — 
2  (e4  -  ei),  or  e4  -  e4  =  £m=i  §  (ejm  -  eim). 

Proof  of  Lemma  3.2.4 ■  Fix  u,v  G  A ,  u  ^  v.  When  K  =  1,  we  can  take  a  sequence 

of  states  u  =  u\,...,ul  =  u,  such  that  ®l(lm.U7n+1  >  0.  Then  we  can  simply  take 

Gy  ~  CU  —  X/m=l  (e«m+ 1  _  eUm) ' 

For  general  K ,  it  is  more  subtle  to  choose  all  the  coefficients  {am}.  By  Definition 
3.2.1,  there  exist  a  sequence  of  distinct  states  u  —  u\,  ...,ul  —  v,  and  km  G  {1, ...,  A'}, 
(im,jm)  G  Jkm  for  m  =  1, ...,  L  —  1,  that  satisfies  the  properties  in  Definition  3.2.1. 
We  claim  that  there  exist  nonnegative  (om  1,  such  that 

L—l 

a ™  (eJm  -  O  =  ~  e“  (3-2-3) 

m=  1  i^u 

with  cf]  >  0,  cf }  >  0. 

To  prove  this  claim,  we  first  let  =  (e„m+1 ,  ejm)  / km  for  m  =  1, ...,  A  —  1.  Since 
by  Definition  3.2.1  um+i  G  jm,  G  [1/A',  1],  Let  {l)fn  I^T'i  be  the  solution  to  the 
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following  system  of  linear  equations: 


< 


K 


Kb 


(u) 

L—l 


(3.2.4) 


K 


+  b 


(u) 

L—2 


+  ...+& 


The  interpretation  of  these  equations  is  as  follows.  Recall  that  by  Definition  3.2. l.ii) , 
for  each  m  =  each  component  of  \m  belongs  to  {ui,...,um}.  Hence  for 

m0  =  2 —  1,  (eUrnQ ,  e;m)  can  be  positive  only  when  m  G  {m0,  ■■■,  L  —  1}.  In 
the  following  display,  the  first  equality  follows  from  the  last  sentence,  the  second 
equality  from  the  definition  of  and  the  inequality  from  (eim,eUmQ)  <  K  and 

the  equations  (3.2.4): 


L—l 


L—l 


m= 1 


1  /,(«).=.  P  \  =  /  ]SU)  p.  _  V'  a(Dp.  p 

Umo  —  1  CJmQ  —  1  /  J  Um  Clm  )  C^mo  j  \  UmQ  —  l°Jmo  —  1  /  v  C1  m'l'^Urr 

\  m=rriQ 

L—l 

-  K(m0-l)i  ilu)  _  flu)  /  \ 

-ft.  ft'm0-lum0_l  /  y  Dn  \eimienmo/ 


m=rrio 


>  o. 


(3.2.5) 


Equality  holds  in  (3.2.5)  only  if  kmQ- 1  =  1  and  if  for  all  m  G  {mo, L  —  1}, 
(eim,  e,imo )  =  K.  Since  equality  for  a  particular  m o  G  {2, ...,  L  —  1}  implies  kmo- 1  =  1 
and  therefore  ( e\  ,,eu  .  \  <  Jl ,  there  must  be  some  index  for  which  the  strict 
inequality  holds.  It  follows  from  (3.2.5)  that 


V  &(“> 

/  v  m 


\m=l 


(ei 


L—l 


/  j  m 

m—  1 


6i„ 


>  0,  for  i  G  {um}TmJ2  ■ 


Also,  for  i  ^  {um}rnJ\ ,  by  Definition  3.2. l.ii),  (e,,  eim)  =  0  for  any  m  =  1, ...,  L.  This 
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implies 


b™ejm,  eA  >  0,  for  i  £  {u™}.^1,  . 

»n=l  / 

The  equality  in  the  following  display  is  because  (ejm  —  e-lm)  (1, . . . ,  1))  =  0,  and  the 
inequality  follows  by  combining  the  last  two  displays  and  using  that  the  first  will  be 
strict  for  at  least  one  index: 


£  ''S’ 


\m=l 


(ef 


=  -£ 

W^U 


<  0. 


The  last  three  displays  imply  there  are  d[u ^  >  0,  i  ^  u  and  d*  >  0  such  that 

L—l 

“  O  =  diU)ei  ~d*en- 

m=  1  ij^u 

To  obtain  (3.2.3)  and  in  particular  the  coefficients  {a™  \  and  {c^}f=li^u,  we 

divide  the  last  display  by  d*.  Then  clearly  c >  0  for  i  ^  u,  and  again  using 
(ejm  -  eim,  (1, . . . ,  1))  =  0  gives  4^  —  1-  To  see  that  c ^  >  0,  we  use  the  last 
display  and  that  v  ^  im  for  any  m,  v  G  jL-i,  and  that  bln  >  0  for  all  m. 


To  obtain  (3.2.2)  we  will  eliminate  the  terms  involving  e,,  i  ^  u,v,  on  the  right 
hand  side  of  (3.2.3).  We  have  assumed  for  each  s  ^  u,v,  that  v  is  K— accessible 
from  s.  Hence  applying  the  same  argument  as  above  with  u  replaced  by  s,  we  obtain 
some  Ls  <  oo,  a  sequence  of  jumps  {(i^,  and  coefficient  (o™  }m=ii  such 


that 


where  cf]  >  0,  4'S)  =  ^  44  >  0,  and  a™  >  0. 


(3.2.6) 
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It  suffices  to  find  nonnegative  {0i}d=]  %^v  such  that 


9l  (^2ci1)ei  _ei)  + - 

V  i/1  /  \i^d 


( d )  _ 

Cj  ej  J  e„  eu, 


(3.2.7) 


since  then  we  coul  substitute  (3.2.6)  into  (3.2.7)  to  obtain  (3.2.2).  Notice  that  (3.2.7) 
can  also  be  written  componentwiseas 


0\  —  +  •  •  •  +  ^0d 

02  =  +  •  •  •  +  cid)0d 


0u  —  1  +  ci  ^0\  +  '  '  '  +  Cu  ^  0d 


0d  ~  Cd  ^01  +  -  -  -  +  <4  ^ 0d- 1- 


Applying  Lemma  3.2.5  below  with 


<  0  cf» 

J1)  •  • 


c  = 


Jd)  \ 


Jd ) 

'd-1 


V 


(1) 


ct1}  0 


and  b  =  eu,  it  follows  that  this  system  of  linear  equations  has  a  unique  nonnegative 
solution.  □ 

Lemma  3.2.5.  Suppose  that  A  =  Id  —  C ,  where  Id  is  the  d  x  d  identity  matrix  and 
C  =  ;  ;  for  some  d  G  N,  such  that  cu  =  0,  c^-  >  0  and  J2j= i  cij  <  1-  A/so,  /et 

5  G  [0,  oo)d .  Then  the  system  of  linear  equations  Ax  =  b  has  a  unique  nonnegative 
solution  {xi}di=l. 


Proof.  The  spectral  radius  of  C  is  less  than  1  since  its  matrix  norm  is  less  than  1. 
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Therefore  det  A  >  0,  and  A _1  exists.  The  fact  that  A is  a  positive  matrix  follows 
from  a  general  result  in  inverse  positivity  [2,  Theorem  6.3.8].  The  nonnegativity  of 
x  then  follows  from  the  nonnegativity  of  b.  □ 

Lemma  3.2.6.  Assume  the  conclusion  of  Lemma  3.2.4  holds.  Then  for  any  a  >  0 
and  x,  y  G  Sa ,  there  exists  a  communicating  path  in  Sa /2  that  connects  x  to  y,  with 
the  number  of  linear  segments  possibly  depending  on  a. 

Proof.  To  start  we  use  the  fact  that  there  is  a  piecewise  linear  path  0 o  from  x  to  y 
that  lies  in  Sa  and  only  uses  velocities  in  the  directions  {ej  —  e,  :  i,  j  G  X},  and  for 
which  the  total  length  scales  with  the  distance  between  x  and  y.  Suppose  0o  is  such 
a  path,  with  0O  (0)  =  x,  and  for  some  L  <  d,  there  are  0  =  t0  <  ti  <  ■  ■  ■  <  tL  and 
{um}t= i ,  ivm}t= i  e  X  so  that 

L 

00  (t)  =  (e«m  ~  O  Mtm-Utm)  (t)  ,  a-e‘  *  E  [0 ,tL\. 

m=  1 

As  noted  previously,  we  can  assume  there  is  C\  <  oo  independent  of  x  and  y  such  that 
f*L  ||0o  (s)  || ds  <  Ci  ||a;  —  y\\.  By  Lemma  3.2.4,  there  exist  Fm  <  oo,  j^)} 

and  {a[]m)}  such  that  eVm  -  eUm  =  Efc=i4m)(ej(m)  _  e.(m)).  Let  F  =  rnaxm  Fm, 
C  =  V 2 KF  maxm)fc  a(fn'> ,  and  set  e  =  a/2C.  We  now  further  divide  each  segment 
tm)  into  finitely  many  segments,  each  of  which  has  length  no  more  than  e,  and 
within  each  segment,  replace  0o  by  a  finitely  segmented  communicating  path.  In 
particular,  if  [rj,r/)  C  is  such  a  segment,  then  we  would  use  the  path 

f‘  S  Fm 

0  (s)  =  00  (r*»)  +  / 

•>ri  k=  1 

Note  that  0(r/)  =  0o  (rf),  and  that  ry  —  rt  <  £  for  all  such  segments  implies  0  G 
AC  ([0,£l]  :  5“/2).  Also,  by  (2.3.3)  and  (2.3.6)  we  have  Xv  (0  (s))  >  Aj  (f)dco  for  all 


(e.w  e.w)  l[n+^rf_riyi+k(rf_ri))  (s)  ds. 
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,s  G  [0,tL],  and 

fL  ||0  (s)  ||ds  <  CF 2  fL  ||0o  (s)  ||ds  <  CF2C\  ||x  -  y\\ . 

Jo  Jo 

Thus  0  is  the  desired  communicating  path  that  connects  x  to  y.  □ 

Now  we  complete  the  proof  of  Proposition  3.1.21.i). 

Proof.  By  Remark  3.2.2,  under  Conditions  3.1.18  and  3.1.20  j  is  K— accessible  from 
i  for  any  i,j  G  X.  Therefore  for  any  x  G  S,  y  G  int(«S),  a  communicating  path 
connecting  x  to  y  can  be  found  by  setting  a  =  min  |t/i,  ...,  y^  {K+lyi-i(l  |,  and  con¬ 
catenating  the  paths  constructed  in  Lemma  3.2.3  and  3.2.4.  Notice  that  along  the 
path  0,  A„(0(s))  >  ^  (f)dc0  >  c1(mmiyi)d,  where  ci  =  ^c0(2  (K  +  l)rf_1  d)~d. 
Therefore  Condition  3.1.4  holds.  □ 

3.2.2  Proof  of  Proposition  3.1.21.ii) 

To  verify  Condition  3.1.14,  for  any  x,y  G  S,  we  need  to  construct  a  strongly  commu¬ 
nicating  path  that  connects  x  to  y.  In  particular,  we  need  to  consider  the  case  when 
y  G  dS.  Thus,  the  approach  in  the  previous  subsection  (in  particular,  Lemma  3.2.6) 
cannot  be  directly  applied.  We  will  show  that  1— ergodicity  implies  a  very  strong 
controllability,  so  that  one  can  always  construct  a  communicating  path  between  x 
and  y  by  matching  up  their  coordinates.  We  first  show  that  Condition  3.1.14  holds 
under  both  Conditions  3.1.18  and  3.1.19,  and  then  show  that  3.1.18  can  be  dropped 
if  T1  is  continuous. 

Lemma  3.2.7.  Assume  Conditions  3.1.18  and  3.1.19  hold.  Then  for  any  x,  y  G 
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S,  there  exist  to  >  0  and  a  strongly  communicating  path  0  G  AC  ([0,  to]  :  S)  that 
connects  x  to  y.  Moreover,  we  can  rescale  0  such  that  0(0)  =  x,  0(1)  =  y,  and 
||0  (•)  ||  is  constant  and  bounded  by  C  ||x  —  y\\  for  some  C  <  oo. 

Proof.  We  will  prove  the  result  by  a  recursive  construction.  We  claim  that  for  any 
r  G  {2,  ...,d},  after  possibly  relabeling  inclicies,  there  exist  0  <  fr_i  <  oo,  some 
z ^  G  S  such  that  z}p  =  yt  for  i  G  {l,...,r  —  1},  and  a  strongly  communicating 
path  0(r)  G  AC  ([0,tr_i]  :  5),  such  that  0^  (0)  =  x,  0^  (fr_ 1)  =  Furthermore, 
fp-1  ||0^  (s)  ||ds  <  Cr  (d,  K)  1 1 a:  —  y\\  for  some  Cr  <  oo.  The  conclusion  follows  by 
taking  r  =  d. 

The  claim  will  be  proved  by  induction.  The  case  r  =  2  is  trivial:  we  choose 
i\  G  X  so  that  xn  >  yH ,  and  then  reverse  the  roles  of  the  inclicies  1  and  i\.  Take 
ti  —  x\  —  yi  and  any  state  i2  G  T\{1}  such  that  911  ]J2  >  0  (the  existence  of  *2  is 
implied  by  Conditions  3.1.18  and  3.1.19),  and  switch  the  inclicies  2  and  *2-  Define 

0(2)  (t)  —  x  +  (e2  -  e0  t,  t  G  [0,  ti] . 

Then  clearly,  0^2i  (0)  =  x ,  =  <f>^  (t±)  G  S,  zP  =  y\  and  J*1  ||0(2i  (s)  ||ds  = 

V2t1  <  y/2  ||x  —  y||.  Thus,  is  a  path  of  the  desired  form.  Also,  the  lower  bound 
(3.1.13)  holds  with  M\  =  {1},  P\  =  1,  and  for  some  c\  >  0  since  by  (2.3.3)  and 
(2.3.6), 

K 

A„i  (x)  =  P2  PP  (x)  >  Xi rj2  (x)  >  c0xi 
k=1  (ij  )eyfc: 

ej  —e\=v\ 

since  Tj2(x)  is  uniformly  positive. 

Now,  assume  the  claim  holds  for  r  =  m,  and  let  and  0(m)  be  the  corre¬ 
sponding  quantities  in  the  claim.  We  now  prove  the  claim  for  r  =  m  +  1.  By 
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assumption,  we  have  q =  z^m\  Since  zf1^  =  Y^,=m  yt,  by  possibly 

indicies  within  the  set  we  may  assume  without  loss  of  generality  that 

>  Urn,  If  Zrnn)  =  y-rn  then  the  claim  also  holds  for  r  =  m  +  1.  If  z,tn)  >  ym , 
we  will  move  some  mass  from  state  m  to  some  state  in  {m  +  1, d},  while  not 
changing  the  mass  in  states  with  lower  index,  by  considering  an  additional  tra¬ 
jectory  ij}  G  AC  ([0,  q]  :  <S)  for  some  q  <  oo,  such  that  ipi(q)  =  ipi  (0)  for  every 
i  =  1,  ...,m  —  1  and  (g)  —  (0)  =  —(zin'1  —  ym )•  To  do  this,  take  any  v  G 

{m  +  1,  ...,d}.  By  Conditions  3.1.18  and  3.1.19,  there  exist  L  <  m  and  a  sequence 
of  states  m  =  uq,...,ul  =  v,  such  that  for  1  <  l  <  L  —  1,  i  >  0.  We  take 

h-n  =  Uimin ,  where  /min  =  min  {I  :  u,  G  {m  +  1, ...,  d}}.  Thus  ulinin  is  the  first  in¬ 
dex  outside  {1,  and  /mjn  is  the  number  of  steps  it  took  to  get  there.  Define 

q  =  /min(>mn)  -  ym) ,  and 


^  (t)  =  z(m) 


pt  ^min 

+  /  y  ^  i&ui  —  eui_i)  i 
i=i 


[(i-i)(4r)-wm),i(*Sr)-wm))  (3-2-8) 


Since  /m;n  <  d,  we  have 


II (s)  || ds  <  V2d  | z^}  -  ym , 


<  (|xm  -  ym|  +  | -  xm\ ) 

Ik  -  2/||  +  /  Ik(m)  (s)  | Ms 

<  v^<i(l  +  Cm  (d,  K))  ||x  -  y\\  , 


where  the  last  inequality  follows  from  the  induction  assumption  for  r  =  m. 

Now  define  tm+ 1  =  tm  +  g,  and  0(m+1l  G  C  ([0,  tm+i]  :  S)  to  be  the  concatenation 
of  and  ij).  As  we  show  below,  is  a  path  of  the  desired  form.  For  v  G  Vi, 

v  =  ej  —  e,  for  some  i,  j  G  A ,  i  ^  j,  as  before  we  have  Xv  (x)  >  xj? ]■  (x)  >  cqXi,  and 
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therefore  for  s  G  [0,£m+i],  A„((/>(m+1)  (s))  >  co0-m+1)  (s).  Finally  we  note  that 

ptm  ptm—1  ptp 

/  u<y+1)wii*  <  /  n#"*>  (»)  ii*  +  /  u{t)\\dt 

Jo  Jo  Jo 

<  (V2d  +  (y 2d  +  lj  Cm  (d,  K)^  ||x  -  y\\ . 

The  claim  now  follows  by  induction.  The  last  sentence  in  the  statement  of  the  lemma 
follows  from  Remark  3.1.6,  thus  completing  the  proof.  □ 

The  construction  of  a  discrete  strongly  communicating  path  follows  directly  from 
discretization  of  a  strongly  communicating  path. 

Lemma  3.2.8.  Suppose  Conditions  3.1.18  and  3.1.19  hold.  Then  for  any  n  G  N, 
i,!/G  Sn,  there  exists  a  discrete  strongly  communicating  path  that  connects  x  to  y. 

Proof.  Fix  n  G  N  and  x,  y  G  Sn.  Let  y—x  =  A  (ai, ..,  af)  with  a,  G  N,  |a*|  <  n  ||a;  —  y\\ 
and  Yli=i  ai  =  0-  By  Lemma  3.2.7  there  exists  a  communicating  path  <f> ^  with 
(j)^  (0)  =  x,  (1)  =  y,  and  U  <  oo  such  that  ||0(c)  (s)  ||  =  U  <  C  ( d,K )  ||x  —  y\\ 
for  almost  every  s  G  [0, 1].  Since  <f>^  only  moves  in  the  directions  {ej  —  :  i,  j  G  X}, 

the  choice  of  the  times  q  used  in  the  recursive  construction  of  0^  in  (3.2.8)  and  the 
fact  that  both  x  and  y  lie  in  the  grid  Sn  imply  that  the  value  0^  at  the  times  when 
0(c)  changes  (denoted  {z^}  in  the  previous  lemma)  are  also  in  Sn .  It  suffices  to 
take  k  =  nU ,  and  the  discrete  path  {0S}^=1  to  be  <fs  =  0^  ( sjnU ),  that  is,  to  be 
the  lattice  point  passed  by  0(cb  Then  0  is  a  path  of  the  desired  form  and  0  satisfies 
(3.1.13)  since  0^  satisfies  (3.1.10).  □ 

We  now  relax  the  assumptions  made  in  the  last  two  lemmas,  and  show  that  they 
continue  to  hold  under  Condition  3.1.19  and  continuity  of  the  jump  rates.  The  basic 
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idea  of  the  proof  is  to  partition  S  into  finite  collection  of  sets  of  essentially  the  same 
geometric  form  as  the  simplex,  such  that  within  each  sets  Condition  3.1.18  holds. 
Therefore,  by  Lemma  3.2.7  and  3.2.8,  one  can  construct  a  strongly  communicating 
path  (and  its  discrete  analogue)  that  connects  any  two  points  within  each  small 
simplex.  The  desired  strongly  communicating  path  (and  its  discrete  analogue)  in 
S  can  be  obtained  by  concatenating  the  paths  within  some  collection  of  the  smalll 
simplices. 

Lemma  3.2.9.  Assume  Condition  3.1.19  is  satisfied,  and  that  T1  (•)  is  continuous. 
Then  the  conclusions  of  Lemma  3.2.7  and  3.2.8  hold. 


Proof.  Recall  that  Vi=  {ej  —  et,  (■ i,j )  G  J1}-  For  any  x  G  S,  let  V(x)  =  {ej  —  e,  G 
Vi  :  Tb  (x)  >  0}  and  cx  =  min^y^)  TL  (x).  By  the  continuity  of  {TL  (•)},  there  exists 
an  open  simplex  Sx  that  contains  x,  and  such  that  inf,ye<sx.  minej_eiey(a;)  Tb  (y)  >  cx/ 2. 
{iSx}x&s  thus  forms  an  open  cover  of  S,  and  we  can  take  a  finite  subcover  {«SXfc}^=1 . 
Let  Co  =  min  {cXl/2, ...,  cXKj 2}.  For  any  x  G  S  let  K(x)  =  {k  :  x  G  and  define 


V\j  (a;)  ei  -  d  e  CkeK(x)V(xk) 


Then  for  any  i  =  1 x  G  SXi,  the  Markov  process  on  X  with  rate  matrices 
{Fb  (x)}  is  ergodic.  This  implies  that  {Tb  (a:)}  satisfies  Conditions  3.1.18  and  3.1.19 
within  SXk,  with  inf yeSxk  mmej_eieVxk  Rb  (y)  >  c0.  Therefore,  by  applying  the  same 
argument  as  in  Lemma  3.2.7,  it  follows  that  for  any  m,dG  SXi,  there  exists  a  strongly 
communicating  path  that  connects  u  to  v. 


Now  for  any  u,v  G  S,  suppose  the  line  {tu  +  (1  —  t)  v  :  t  G  [0,1]}  intersects 
U k=1dSXi  at  {ui}f=l,  for  some  L  <  oo.  Then  each  line  segment 
{tut  +  (1  —  t)  ut+\  :  t  G  [0, 1]}  is  contained  in  some  <SXj,  and  therefore  one  can  take  a 
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strongly  communicating  path  fa  that  connects  Ui  to  ul+ 1 .  Let  0  be  the  concatena¬ 
tion  of  Then  0  is  strongly  communicating  because  each  fa  is.  Indeed,  the 

fact  that  each  fa  satisfies  Condition  3.1.4.i).ii).  and  Definition  3.1.12.iv).v)  directly 
implies  0  does.  Also,  denote  uq  =  u,  ul+i  =  v,  we  have 

L  L 

Len(fa)  <  Len(fa)  <  C  ||wj+i  —  Ui\\  —  C  \\u  —  u|| , 

2—0  2=0 

for  some  C  <  oo,  where  we  used  the  fact  that  each  fa  satisfies  Condition  3.1.4.iii). 
Therefore  0  satisfies  Condition  3.1.4.iii). 

The  construction  of  a  discrete  strongly  communicating  path  follows  by  exactly 
the  same  argument  as  Lemma  3.2.8.  □ 

3.2.3  Proof  of  Lemma  3.1.9 

As  before,  we  first  illustrate  the  conclusion  with  Example  3.1.25  where  d  —  4  and 
K  =  2.  Let  x  —  n  (0),  and  assume  without  loss  of  generality  x\  >  X2  >  x%  >  x±,  and 
therefore  X\  >  1/4.  The  ODE  (2.3.2)  implies 

/li  =  y,  {v,  ei)  A„  (n  (t))  >  — yUi  -  H!ix2  >  -2/7i, 

vev 

thus  fi i  (t)  >  x\e~2t  >  Co,  Co  =  e~2/4.  Then 

lh  =  Y,  ( V ,  e2)  A„  (/x  (£))  >  HI  -  -  H1H2  >  Co  -  2/x2, 

•yev 
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which  one  solves  /12  (t)  >  c.\t  for  some  c\  >  0.  Also,  for  i  —  3,4, 

/ij  ^  (v ,  e*)  A„  (/1  (t))  A  1/^2  /T  h'3h,4  ^  c0cit  2/ij, 

^ev 

thus  /i,  (t)  >  C2t2.  The  proof  for  the  general  case  is  more  technical  and  is  given 
below. 

Proof.  Define  61  =  max  { |  (e^ ,  v)  :  j  e  X,  r  6  V}  <  00,  and  let  Mi  =  b\M\V\.  The 
ODE  (2.3.2)  implies  that  for  i  —  1, ...,  d, 

fk  ( t )  =  y;  (t>,  e0  A„  (11  (t))  >  —Ml,  t  6  [0, 1]  .  (3.2.9) 

^ev 

Therefore 

fk  (t)  >  Ik  (0)  e~Mlt,  t  e  [0, 1]  .  (3.2.10) 

Let  X\  =  \i  G  A  :  /i*  (0)  >  Tj.  Then  for  any  b  <  T e~Mi ,  (3.2.10)  implies  that 
/i,  (£)  >  b,  t  e  [0, 1]  holds  for  all  i  G  thus  Condition  3.1.3  is  satisfied. 

To  show  the  inequality  in  Condition  3.1.3  also  holds  for  %  €  A\A’1  (for  a  suitable 
choice  of  b),  the  idea  is  that  in  (3.2.9),  for  those  v  such  that  {v^ef)  <  0,  Xv(x) 
converges  to  zero  as  Xi  — ►  0  (by  Condition  3.1.7),  and  therefore  the  communication 
condition  would  push  y  (•)  into  the  interior  of  S.  Define  y  =  Q, ...,  A),  and  note  that 
by  Condition  3.1.4  and  Remark  3.1.6,  there  exist  c  1  >  0,  and  a  communicating  path 
0  e  AC  ([0, 1]  :  S)  with  0(0)  =  y(0),  0(1)  =  y.  Given  the  representation  of  0  in 
terms  of  F  <  00 ,  (fmlLi ,  WLi  and  Siven  in  (3-1.2),  property  ii)  of 

Condition  3.1.4  implies  for  m  =  1, ...,  F, 


A vm  (0(s))  >ci,se[t  m— 1 7  trri\  • 


(3.2.11) 
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For  m  =  1,  denote  =  (f>(tm).  Then  in  particular,  we  have  A„m+1  > 

ci.  Using  Condition  3.1.1,  there  exist  C'2,  C3  >  0  such  that  the  following  lower  bound 
holds  for  any  z  G  S: 


Aum+1 (2) 


A 


Pm+ 1 


X 


M) 


> 


^n^rexp  (-/o  (h(m>  -  ‘ii»  >  c3n 


i=i  xi 


3=1  X3 


(m)  ’ 


(3.2.12) 


where  the  last  inequality  follows  from  the  fact  that  /o  (•)  is  continuous  (and  thus 
bounded  from  above)  on  [0,1]. 


Now,  define  Vk  =  {j  G  X  :  /q  (0)  <  } ,  and  fix  i  G  T\T’i.  Clearly,  i  G  Pf 

since  x^  =  y*  =  l/d.  We  claim  that  for  every  k  =  1,...,P,  there  exists  b ^  >  0 
such  that  for  any  j  G  Vk,  /q  (£)  >  b^kHD^k\  where  D  (/c)  =  d\  Since  i  G  Pf, 

the  lemma  follows  from  setting  k  =  F  and  b  =  in  the  claim. 


To  prove  the  claim,  we  first  note  that  by  (3.2.10),  for  any  m  =  1  ,...,F  and 
j  Vm,  Hj  (; t )  >  e~MlHj  (0)  >  e~MlXjU\  For  t  >  0,  applying  (3.2.12)  to  obtain 


^m+1  (g  (*)) 

A,m+1  (*(m)) 


>  CV-^l  n^>  Gae-"'"  ]J  ft  W  ■ 

iePm  xj  iePm 


(3.2.13) 


Notice  that  if  for  some  j  G  Vm ,  x^m)  =  0,  the  second  inequality  in  (3.2.13)  also  holds 
trivially. 


We  now  prove  the  claim  by  induction.  Define 
&2  =  nrin  {(u,  ej)  :  j  G  X,  v  G  V,  s.t.  (v,  e,)  >  0}.  For  k  —  1,  take  any  j  G  Pi.  Since 
Hj  (0)  <  xj1^,  we  have  (iq ,ej)  >  0.  By  (2.3.2),  (3.2.11)  and  Condition  3.1.7, 

A?  (*)  >  (vi,  ej)  K  {n  it))  +  ei)  (*)) 

v£V 

(v,ej)<0 


>  b-jci  —  .\I>iij  ( t ) 
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for  M2  =  b  1 C  |V|,  where  C  is  the  constant  in  Condition  3.1.7.  Applying  the  compar¬ 
ison  principle  for  ODEs,  we  see  that  for  t  G  [0, 1],  fi-j  (t)  >  (l  —  e_Mz* )  >  b^H 
for  some  b^  >  0. 


Assume  the  claim  holds  for  k  <  m,  and  let  k  =  m+ 1.  For  l  G  Vm+\ ,  since  Ah  (0)  < 
x[m+1)  =  &  (tm+ 1),  and  <p  has  representation  (3.1.2),  there  exists  m*  G  {1, ...,  m  +  1}, 
such  that  (vm*,ei)  >  0,  and  therefore  (vm*,ei)  >  b2.  If  m*  =  1,  the  result  follows 
by  applying  the  same  argument  in  the  k  =  1  case.  Note  that  by  (3.2.13)  and  the 
induction  hypothesis  for  k  =  m,  there  exist  C\  >  0  such  that  when  m*  >  2,  for 

te  [0,1], 


^vm*  (h  (^)) 

A„„.  (x'"*'-1') 


>  C3e-M'd  JJ  ,1,  it)  >  ('■,<  u"‘  (//’"'  v 

>  cjz&r1*. 


Thus,  combined  with  (2.3.2),  (3.2.11)  and  Condition  3.1.7,  there  exist  C5  >  0  such 
that 


Ah  (t)  >  ( vm * ,  ei)  A„m,  (jj,  (t))  +  ( v >  e*>  (k-  (*)) 

t>ev 

>  C5t^ldi 

>  (A). 

Solving  the  ODE  ii  (t)  =  —M2u  (t),  and  applying  the  comparison  principle, 
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it  follows  that  for  t  G  [0, 1], 


G? 

IV 

^5  -M2t  . 

-i)E: 

Mfi=0 

> 

M2t  , 

-i)E 

Mfi=0 

> 

Jj(m+ 

m  t  i 

i=0  a 


L 
L 


SYT=  id'e  Sds 


s^dids 


for  some  b^71+1'1  >  0.  This  proves  the  claim,  and  hence  the  lemma. 


□ 


3.3  The  Variational  Representation  Formula 

3.3.1  Variational  Representation  for  a  Poisson  Random  Mea¬ 
sure 

We  briefly  review  the  variational  representation  formula  for  a  Poisson  random  mea¬ 
sure  stated  in  [6].  For  any  Polish  space  S  let  Ma  (. S )  denote  the  space  of  a— finite 
measures  on  S.  We  equip  Ma  (. S )  with  the  weakest  topology  such  that  for  every 
/  G  Cc(S),  the  function  u  »  Jsfdu,u  G  Ma(S),  is  continuous.  Let  y  =  [0,  oo), 
At  =  [0,  T]  x  y,  both  equipped  with  the  usual  Euclidean  topology,  and  let  M.  = 
For  some  fixed  measure  v  G  Ma  (A),  let  vt  =  my®!/,  where  mx  is  Lebesgue 
measure  on  [0,T],  For  6  G  [0,  cxd),  let  Pg  denote  the  unique  probability  measure  on 
(AT  B  (M  j)  under  which  the  canonical  map  N  :  M.  — >  A4,  N  (u)  =  u,  is  a  Poisson 
random  measure  with  intensity  measure  9vt-  Let  denote  expectation  with  respect 
to  P#.  For  notational  convenience,  we  omit  the  dependence  of  Pg  and  Eg  on  the  fixed 


measure  vt- 
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Now,  define  a  controlled  Poisson  random  measure  as  follows.  Let  W  =  y  x 
[0,  cx))  and  Wt  =  [0,  T]  x  PP  =  At  x  [0,  oo),  also  equipped  with  the  Euclidean 
product  topology.  Let  AA  =  Mc{Wt)  and  let  P  be  the  unique  probability  measure 
on  (M,B  (At))  under  which  the  canonical  map  N  :  AA.  — >  At,  N  ( u)  =  u,  is  a 
Poisson  random  measure  with  intensity  measure  V t  =  vt®  m,  where  m  is  Lebesgue 
measure  on  [0,  oo).  Let  E  denote  expectation  with  respect  to  P.  Also,  define 

Qt  —  o  {A-  ((0,  s]  x  A)  :0  <s<t,AeB{W)}, 

and  let  Tt  denote  its  completion  under  P.  Denote  by  V  the  predictable  cr-field  on 
[0,T]  x  Ai  with  the  filtration  {A}G<t<T  on  (At,B  (A4)). 

Definition  3.3.1.  Let  A  be  the  class  of  {V  ®  B  (3^))  \£>[0,  oo)  predictable  maps  <p  : 
[0  ,T]xAtxy^  [0,  oo). 

The  role  of  p  is  to  control  the  intensity  of  jumps  at  (s,  u,  x)  by  thinning  in  the  r 
variable.  For  ip  E  A,  define  Nv  :  Ai  — >  Ai  by 

p  poo 

N£((0 ,t\xU)=  /  l[o ,<p(s,u>,x)\  (r)  Nu  (dsdxdr) ,  (3.3.1) 

J(0,t]xU  Jo 

for  t  e  [0,  T) ,  U  G  B  ((P) ,  u  e  M.  We  will  suppress  the  dependence  of  p  (t,  u,  x ),  Nw 
and  Nf  on  to  at  later  times.  Then  under  P.  Nv  is  a  controlled  random  measure  on 
(Pt  with  p  (s,  x)  determining  the  intensity  for  points  at  location  x  and  time  s.  With 
some  abuse  of  notation,  for  9  G  [0,  oo)  we  will  let  Ne  be  defined  as  in  (3.3.1)  with 
p  (s,x)  =  9.  Note  that  the  law  (on  J\A)  of  Ne  under  P  coincides  with  the  law  of  N 
under  P#. 
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Recall  t  (•)  as  defined  in  (3.1.4).  For  ip  G  A  define  the  random  variable  Lt  (ip)  by 

Lt  {ip)  (u>)  =  /  £(<p(t,u,x))  vt  (dtdx) 

JyT 

—  f  I  £  (ip  (t,  u,  x))  v  (dx)  j  dt,  lu  G  M.  (3.3.2) 

Jo  \J[  0,oo)  / 

Definition  3.3.2.  Define  Ab  to  be  the  class  of  (P  ®  B  ((V))  \<B[0,  oo)  predictable 
maps  ip  such  that  for  some  B  <  oo,  ip  (t,  u,  x)  <  B  for  all  ( t ,  u,  x )  G  [0,  T]  x  M.  x  y . 

In  later  sections  we  will  set  T  =  1,  and  hence  the  dependence  of  A  and  At, 
on  T  can  be  omitted.  Let  Mf}  (At)  denote  the  space  of  bounded  Borel  measurable 
functions  on  M..  We  then  have  the  following  representation  formula  for  Poisson 
random  measures. 

Theorem  3.3.3.  Let  F  €  Mb  (M).  Then  for  any  6  >  0, 

-  log [exp  (-F  (IV))]  =  inf  E  [9LT  (ip)  +  F(N0V)]  .  (3.3.3) 

PSA 

Proof.  For  F  G  Mb  (Al)  and  9  >  0,  it  follows  from  Theorem  2.1  of  [6]  that 

—  logEe  [exp  (— F  (A))]  =  —  logE  [exp  (— F(1V61))] 

=  infE  [9Lt  M  +  F(N^)]  ■ 

Moreover,  Theorem  2.4  of  [5]  states  that  the  above  infimization  can  in  fact  be  taken 
over  the  smaller  class  of  controls,  such  that  ip  ( t,u,x )  is  bounded  by  B  <  oo  for  all 
(t,  u)  G  [0,  T]  x  M.  and  for  all  x  within  some  compact  set,  and  ip  (t,  u,  x)  is  identical 
to  1  for  x  outside  the  set.  Since  Ab  C  A  contains  this  class  of  controls,  we  obtain 
(3.3.3).  □ 
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3.3.2  Variational  representation  for  the  empirical  measure 
process 

In  this  section  we  derive  a  variational  representation  formula  for  the  empirical  mea¬ 
sure  process  /jn .  We  represent  /in  as  a  solution  to  a  stochastic  differential  equation 
that  is  driven  by  finitely  many  i.i.d  Poisson  random  measures,  and  using  thinning 
functions  to  obtain  the  desired  jump  rates.  We  then  derive  a  variational  representa¬ 
tion  formula  for  /in,  by  viewing  it  as  the  image  of  a  measurable  mapping  that  acts 
on  the  collection  of  rescaled  Poisson  random  measures. 

Take  v  =  m,  so  that  ut  =  tht  ®  m.  For  n  G  N,  let  {IV™,  v  G  V}  be  a  collection  of 
i.i.d  Poisson  random  measures  (on  At)  with  intensity  measure  rwr ■  Thus  we  have 
the  following  SDE  representation  for  the  empirical  measure  process:  for  t  G  [0,  T\, 

»n(t)  =  nn( +  [  /  1[0,a »frnm(x)-N2(dsdx).  (3.3.4) 

J[o,t)Jy  n 

The  existence  of  a  solution  to  (3.3.4)  is  explained  in  the  following  argument. 

From  Condition  2.3.1  and  (2.3.1),  we  can  set 

M  =  sup  A™  (x)  <  oo.  (3.3.5) 


Let  Aiatom  denote  the  set  of  all  m  =  {rn„,  v  G  V},  where  for  each  v  G  V,  m,,  is  an 
atomic  measure  on  ,  with  the  property  that  m„ ({t}  x  [0,  M'])  >  0  for  only  finitely 
many  t.  Define  h  :  A iatom  X  S  x  ([0,  cx))5")0'V'  — >  D  ([0, 1]  :  S )  as  the  mapping  that 
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takes  (m,  p,  Xn)  G  Adatom  x  S  x  ([0,  oo)5")0^  to  the  process  y  defined  by 


y(t)  =  P+22v  /  1[0,A  n(y(s))](x)mv(dsdx). 


(3.3.6) 


The  existence  of  a  solution  y(-)  to  (3.3.6)  is  easily  verified  by  the  following  re¬ 
cursive  construction.  Set  to  =  0,  and  define  y°  (t)  =  p  for  t  >  0.  Assume  as  part 
of  the  recursive  construction  that  for  some  k  G  No,  a  solution  yk  (•)  to  (3.3.6)  has 
been  constructed  on  the  interval  [0,  tk],  and  that  yk  (t)  =  yk  (tk)  for  t  >  tk ■  For  any 
t  G  [tk,  T]  and  v  G  V,  let 

Av{t)  =  {(s,i):sG  [tk,t],xE  [0,A£  (/(s))]}, 


and 

tk- i-i  =  inf  {t  >  tk  such  that  for  some  v  G  V,  m„  (Av  (t))  >  0}  A  T. 

We  then  define  yk+1  :  [0,  T]  — >  Rd  by  setting  yk+1  (t)  =  yk  (t)  for  t  G  [0,  ffc+i), 

Vk+1  (tk+ 1)  =  yk  (tk)  +  y^v  /  lf0 k,  }\Jx)mv(dsdx), 

^  J[tk,tk+1]Jy  lu’A^y(sbJ 

and  setting  yk+1  (t)  =  yk+1  (tk+1)  for  t  G  [tk+ 1,  T\. 

Since  rn„  has  finitely  many  atoms  on  [0,  T ]  x  [0,  M'],  the  construction  will  produce 
a  function  defined  on  all  of  [0,  T]  in  L  <  oo  steps,  at  which  time  we  set  y  (t)  =  yL  (t). 
Since  N  G  Adatom  for  Pn— a.e.  u>,  we  can  write 

/un(t,  u)  =  h\-Nn,nn(t),u),\n 
\ri 


0 t ) 


(3.3.7) 


for  Pn— a.e.  u  G  Ad. 
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We  now  describe  two  classes  of  controls  that  will  be  used  below.  Recall  that 
V t  =  vt  ®  m.  Let  {N™,v  G  V}  be  a  collection  of  i.i.d  Poisson  random  measures  on 
Wt  with  intensity  measure  twt-  Let 

Q?  =  a  {Nn  ((0,  s]  x  A)  :  0  <  s  <  t,  A  e  B  (y)  } 

and  let  T™  denote  its  completion  under  P.  Denote  by  Vn  the  predictable  cr-field 
on  [0,  T]  x  M  with  the  filtration  {A™  :  0  <  t  <  T}  on  [M,  B  (Wl)) .  For  n  G  N,  we 
denote  Afn  and  A®n  as  the  n— fold  Cartesian  product  of  A,  and  A.  Given  (p  G  A V' 
we  define  the  controlled  jump  Markov  process  fin  G  D  ([0,  T]  :  S )  to  be  the  solution 
to  the  following  SDE:  for  t  G  [0,  T], 


(<)  =  ^(0)  +  £>  [  [ 


__  1[o,a»(A"(»))](®)  /  L[o ,Vv(s,x)](r)~NZ(dsdxdr). 

gV  J[Q,t)Jy  J[  0,oo)  n 

(3.3.8) 


As  described  previously,  (pv(s,  x )  will  control  the  jump  rate  as  a  function  of  (s,  u,  x ). 
In  particular,  the  overall  jump  rate  is  the  product  A”  (/t”(s))  (pv(s,  x),  so  that  (pv(s,  x) 
perturbs  the  jump  rate  away  from  that  of  the  original  model.  For  »  G  V,  let  N 
be  defined  as  in  (3.3.1),  with  tp  replaced  by  <pv  and  N  replaced  by  TV",,  and  let 
Nntf>  =  {N^,v  G  V}.  For  fixed  ip  G  AfM,  Nn*  G  Matom  a.s.  From  the  definition  of 
h(-),  it  is  clear  that  (3.3.8)  is  equivalent  to  the  relation 


jT  =  h  0),Ar 

n 


Applying  Theorem  3.3.3  and  (3.3.7)  we  obtain  the  following  representation  for¬ 
mula  for  /in . 
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Lemma  3.3.4.  For  F  €  Mb  ( S ), 


n 


logE  [exp (— nF(/i”))] 


inf  E 


V  It  (*>„)  +  F(A")  :  A” 


Lt>eV 


h  ^-Nn,fi,  nn(0),  \n^j 


Proof.  Consider  the  space  y  =  y  x  V  and  define  a  collection  of  Poisson  random 
measures  {N71}  on  y  with  intensity  measure  twt  ®  |-|,  where  [•[  is  the  counting 
measure  on  V.  Then  the  SDE  representation  (3.3.4)  is  equivalent  to 


fin(t)  =  fin( 0)  +  [  fvl [0tx^(s))}(x)-Nn(dsdxdv),  te[0,T}. 

J[o,t)  Jy  n 

Therefore  for  any  F  £  Mb  ( S ),  applying  Theorem  3.3.3  with  y  replaced  by  3b  v 
replaced  by  rwr  ®  |-|,  F  (•)  replaced  by  nF  o  h  (i-,  /j,n( 0),  An)  yields 


logE  [exp(— nF(/in))]  =  —  —  logE 
n  n 


=  inf  E 


exp  -nF  o  h  (  -Nn,fj,n( 0),An 


n 


^  Lt  (<pv)  +  Foh[  - Nn /x"(0),  Ar 


□ 


We  now  derive  a  simpler  form  of  the  variational  representation  formula  than  the 
one  given  in  Lemma  3.3.4.  The  starting  point  for  Lemma  3.3.4  is  the  representation 
given  in  [6],  which  is  general  enough  to  cover  situations  where  the  different  points 
in  y  correspond  to  different  “types”  of  jumps.  For  our  purposes  this  is  in  fact  more 
general  than  we  need,  since  all  points  in  3^  correspond  to  exactly  the  same  type  of 
jump,  and  all  that  is  needed  from  the  space  3^  is  that  it  be  big  enough  that  arbitrary 
jump  rates  [such  as  A"  (/fn(s))]  can  be  obtained  by  thinning.  Because  all  the  ads 
play  an  identical  role,  one  expects,  and  we  will  verify  using  Jensen’s  inequality,  that 
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one  can  restrict  to  controls  with  no  ^-dependence.  Thus  we  will  replace  the  time, 
state  and  v  dependent  controls  A^''  by  controls  A^''  that  only  have  time  and  v 
dependence,  and  rewrite  the  running  cost  as  a  function  of  the  new  controlled  jump 
rates. 

Definition  3.3.5.  Define  Ab  to  be  the  class  of  V\B[ 0,  oo)  finite  measurable  maps 
<p  :  [0,  T]  x  At  — >  [0,  cx)). 

Define  An  :  M®|V|  x  S  ->  D  ([0, 1]  :  S)  by 

An  («,P)  (t)  =  P+  'V'v  [  f  l[0,av(s)]A)-N™(dsdx).  (3.3.9) 

veV  J[o,t)Jy  n 

A”  (-,p)  is  well-defined  for  a  G 

We  are  now  in  a  position  to  state  the  main  variational  representation  formula, 
the  proof  of  which  is  deferred  to  Appendix  B.  The  representation  is  the  right  one  for 
finite  state  Markov  chains,  and  expresses  the  variational  integrand  as  the  sum  of  a 
cost  for  perturbing  jump  rates,  plus  the  expected  value  of  the  test  function  evaluated 
at  the  process  which  uses  these  perturbed  rates. 

Theorem  3.3.6.  Let  F  €  Mb  ( S ).  Then 

—  —  logE  [exp (—nF(pn))} 
n 

=  inf  E  V  /'  \nv  (, pn(t ))  i  ( dt  +  F(A)  ■  A  =  A”  {a,  A  (0))  . 
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3.3.3  The  Law  of  Large  Numbers  limit 

We  prove  the  law  of  large  numbers  limit  (Theorem  2.3.2)  at  the  end  of  this  section. 
First  recall  the  law  of  large  numbers  result  for  scaled  Poisson  random  measures:  for 
any  A  G  B  ([0, 1]  x  [0,  oo))  such  that  m\  ®  m  (A)  <  oo,  A  TV"  (A)  — >  m\  ®  m  (A)  in 
probability,  for  any  v  G  V.  This  implies  that  for  any  f  E  Cc  ([0, 1]  x  [0,  oo))  we  have 
fo  r  f  (s’  x)  n^v  ( dsdx )  — »  Jg1  r  f  (s>  x)  dsdx.  Rewrite  (3.3.4)  as 

nn(t)  =  po + v  \  A”  o^)) ds + Mn(*)> 

vev  Ji°A 

where  Mn(t )  =  fo  fo°°  ilo, a %{nn(s))](x)(f,N™(dsdx)  -  dsdx).  Recall  that  A”  con¬ 

verges  uniformly  to  A„  (by  Condition  2.3.1).  Therefore  if  for  any  £  >  0, 

P(supte[0T]  ||Mn(t)||  >  e)  — ■>  0,  then  pn  — >  p  in  probability  (uniformly  on  t  G  [0,T]), 
for  some  p  which  satisfies 

fi{t)  =  po +  yfv  A  v(p(s))ds, 

vev  Ji°A 

which  is  the  integral  version  of  (2.3.2).  The  uniqueness  of  the  solution  p(-)  to  the 
above  equation  follows  from  the  Lipschitz  continuity  of  A„  (•). 

Since  {Mn(t),t  >  0}  is  an  {J7™}  —martingale,  for  any  £  >  0  Doob’s  inequality 
gives 
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P  sup  \\Mn(t)W  >  £ 

\ie[0,T] 

<  -4EiiMn(T)ii2 


< 


£2 


E 


r*T  POO 


|V| 


nez 


'1  nM 


'  /  1[0,A n(,j,n(s))](x)-(N™(dsdx) -ndsdx) 

0  Jo  n 


E 


dsdx 


o  Jo 


as  n  — >  oo,  where  M  is  defined  as  in  (2.3.1).  This  completes  the  proof. 


3.4  Proof  of  the  LDP  Upper  Bound 


A  large  deviation  upper  bound  for  a  general  class  of  Markov  processes  was  obtained 
in  [13] .  We  briefly  describe  the  results  in  [13]  below,  specialized  to  the  current  setting. 
For  every  n  e  N,  recall  the  Markov  process  /in  on  Sn  with  infinitesimal  generator  Cn 
given  by  (2.2.5).  Theorem  1.1  of  [13]  applies  to  a  slightly  different  class  of  Markov 
processes,  with  infinitesimal  generator  described  by 


(/)  (®)  =n'y\v  (x) 


v&> 


f  (x  +  -v)  -  f(x) 
n 


where  A,;  takes  values  in  (2.3.6). 


(3.4.1) 


The  difference  between  (2.2.5)  and  (3.4.1)  is  the  n— dependence  of  jump  rates. 
However,  since  A™  convergences  uniformly  to  A,,,  Theorem  3.4.1  below  implies  that 
these  processes  have  the  same  LDP  rate  function.  The  proof  follows  by  a  standard 
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coupling  argument,  and  will  be  deferred  to  Appendix  C. 

We  assume  Condition  2.3.1  for  the  results  in  this  section.  In  what  follows  we 
denote  ||a;  —  =  supter0u  ||a;  (t)  —  y  (f)||  for  x,y  G  D  ([0, 1]  :  5).  Recall  that  two 

squences  of  Markov  processes  {An}ngN  ,  {Yn}  N  cD([0,l]  :  S)  are  called  exponen¬ 
tially  equivalent,  if  for  each  £  >  0, 

limsup  —  logP(||Xn  —  T”||  >  e)  =  — oo. 

n— >oo  Tl 

Exponential  equivalence  implies  that  {Xn}ngN  satisfies  the  LDP  with  a  given 
rate  function  if  and  only  if  {En}nepj  satisfies  the  LDP  with  the  same  rate  function 
(Theorem  1.3.3  of  [12]). 

Theorem  3.4.1.  Assume  the  family  of  jump  rates  {A„  (x) ,  x  G  S,  v  G  V}  satisfies 
Conditions  2.3.1.  Let  {A"n}ngN ,  {Tn}ngN  be  a  squence  of  Markov  processes  with 
generator  Cn  and  C[\  respectively,  and  with  Xn  (0)  =  Yn  (0).  Then  { Xn }  and  { Yn } 
are  exponentially  equivalent. 

For  x,  a  G  define 

H  (x,  a)  =  A,;  (x)  (exp  (a,  v)  —  1) . 

uev 

Note  that  H  is  continuous.  Let  L°  be  its  Legendre-Fenchcl  transform  defined  by 

L°  (x,  /3)  =  sup  [(a,  f3)  —  H  (x,  a)] .  (3.4.2) 

ag  Rd 

Also,  for  t  G  [0, 1]  define  /t°  as  in  (3.1.5),  but  with  L  replaced  by  L°. 
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Proposition  3.4.2.  For  any  compact  set  K  C  S  and  M  <  oo,  the  set 

{7:/0  (7)  <M,  7(0)  eK} 

is  compact.  Assume  the  family  of  jump  rates  {A„  (x) ,  x  G  S,v  G  V}  satisfies  Condi¬ 
tions  2.3.1.  Also,  assume  that  the  initial  conditions  {/in(0)}ngN  are  deterministic, 
and  pn  (0)  /i0  6  P  (X)  as  n  tends  to  infinity.  Let  {Pn}ngN  be  a  sequence  of 

Markov  processes  with  generator  C[\,  and  Yn  (0)  =  p"'  (0).  Then  {Yn}  satisfies  the 
large  deviation  upper  bound  with  rate  function  1° . 

Proof.  This  follows  from  Theorem  1.1  of  [13]  with  e  =  1/n,  «(•)  =  &(•)  =  0,  and 

px  (■)  lzGcS  ^ v  (*^)  (')•  I— I 

We  have  introduced  functions  L°  [in  (3.4.2)]  and  L  [in  (3.1.3)],  defined  respec¬ 
tively  in  terms  of  a  Legendre  transform  and  relative  entropy.  The  next  proposition 
shows  they  are  two  representations  of  the  same  local  rate  function. 

Proposition  3.4.3.  L  (x,  f3)  =  L°  ( x ,  (3 )  for  all  x  G  S,/3  G  Ad_1 . 

Proof.  Defining  hv,a  :  — >  M  by  hv,a  (a)  =  a  (exp  ((a,v))  —  1)  for  v  G  and  a  G 
[0,  cx)),  we  can  write  H  (x,a)  =  Y2vev  ^V,xv(x)  (a;).  The  Legendre-Fenchel  transform 
of  hVta  can  be  computed  explicitly  as 

,  *  /  n\  ai(y^  liP  =  aVVi 

K,a  (P)  =  \ 

j  00  otherwise. 

Since  H  is  a  finite  sum  of  convex  functions,  we  can  apply  a  standard  result  in  convex 
analysis  to  calculate  its  Legendre-Fenchel  transform  (see,  e.g.,  Theorem  D.4.2  of 
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[12]): 


Hence, 


(P)  =  inf 


{Pv)  ■  ^2  Pv 

vev 


L°(x,l3 ) 


inf 

vQv=0 


y  xv  {x)  t 

vdV 


L(x,P). 


□ 


Combining  Theorem  3.4.1,  Proposition  3.4.2  and  Proposition  3.4.3,  we  conclude 
that  that  I  (7)  satisfies  (3.1.7),  and  has  compact  level  sets  for  compact  sets  of  initial 
conditions. 


3.5  Properties  of  the  Local  Rate  Function 

In  this  section  we  establish  useful  properties  of  the  function 

L{x,p)  =  inf  y  Xv  (x)£  ,  x  E  S,P  e  Ad_1,  (3.5.1) 

which  is  defined  in  (3.1.3)  as  a  proposed  local  rate  function  for  {pn}neN-  In  what 
follows  we  denote  the  relative  interior  of  S  by  int(«S).  Also,  for  a  >  0,  define 

Sa  =  {p  G  S  :  dist  (p,  dS )  >  a } 

=  <  p  G  S  :  inf  ||p  —  x\\  >  a  >  .  (3.5.2) 

I ’  x£dS  j 

Note  that  for  x  E  Sa ,  Xi  >  a/\fd  for  i  =  1, ...,  d. 


Throughout  this  section,  we  assume  Conditions  2.3.1,  3.1.1  and  3.1.4  hold  for 
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{ A,,  (-)}„ev  Given  a  set  of  vectors  {uj}J=1  C  Md,  the  positive  cone  spanned  by 
{uj}J=  i  is  defined  by 


C iuj} ~ 


v  e 


id  :  there  exist  a,j  >  0  with  v  = 

3= 1 


ajuj 


(3.5.3) 


Also,  define  Vx  —  {v  G  V  :  A„  (x)  >  0}, 

V+  =  {v  G  V  :  for  any  a  >  0,  infxe<sa  (x)  >  0}.  Under  Condition  3.1.1,  for  x  G 
int(«S),  Vx  =  V+.  The  following  observation  follows  from  the  definition  of  a  commu¬ 
nicating  path. 

Proposition  3.5.1.  For  x  G  int(«S),  C  {v  :  v  G  Vx}  =  C  {v  :  v  G  V+}  =  Ad_1. 


ft  is  also  convenient  to  introduce  another  function  L  :  [0,  oo)lvl  x  Ad  1  — >  M  which, 
for  any  vectors  u  G  [0,  oo)  and  j3  G  Ad_1,  is  defined  by 


L  (u,  (5) 


(3.5.4) 


Lemma  3.5.2.  L  is  jointly  strictly  convex  in  u  G  [0,oo)lvl  and  j3  G  Ad_1.  The 
function  L  defined  in  (3.5.1)  is  nonnegative  and  uniformly  continuous  on  compact 
subsets  of  int(iS)  x  Ad_1,  and  for  each  x  G  S ,  L  (x,  •)  is  a  strictly  convex  on  Ad_1. 


Proof.  The  joint  convexity  of  the  function 

(u,  q )  G  [0,  oo)  x  [0,  oo)  uv£  ( qv/uv )  is  immediate.  The  joint  convexity  of 

L  then  follows  since  it  is  the  inhmum  of  a  jointly  convex  function  subject  to  an  affine 
constraint.  To  show  that  L  (x,  •)  is  strictly  convex,  note  that  q  i— »  J2vevUv^  (Qv/uv) 
is  strictly  convex,  and  goes  to  infinity  as  ||g||  — >  oo.  Thus  the  minimum  on  a  closed 
convex  set  is  uniquely  attained:  given  any  u  G  [0,  oo) and  j3  G  A d~1,  there  exists 
q*  (/3)  such  that  vq*  =  (3,  and  L  (■ u ,  (3)  =  J^^gy  uvi  (q*/uv).  Therefore  for  any 
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(i\  ^  (h  and  <5  G  (0, 1), 


5L(u,p1)  +  (l-5)l(u,p2)  =  6j2uvi  +(1  -S)J2uv£ 

t>sv  '  v  )  t>£V 

>  u  bq.1  (A)  +  (1  —  j)  0.1  (Pz)  \ 

vev  \  uv  ) 

>  L(u,SPi  +  (1  -  S)p2)  • 


The  nonnegativity  of  L  follows  directly  from  the  definition  (3.5.1).  For  any  a  >  0, 
by  Remark  3.1.2,  A„  (•)  is  either  identically  zero  or  uniformly  bounded  below  away 
from  zero  on  Sa.  Fix  any  j3  G  Ad_1.  By  Proposition  3.5.1,  there  exists  q  G  [0,  oo)lvl 
such  that  Ylvev  vqv  =  /?,  and  qv  —  0  for  v  G  V\V+.  Since  A„  (•)  is  continuous,  it 
follows  that  L  is  uniformly  continuous  on  compact  subsets  of  int(<S)  x  Ad_1.  The 
strict  convexity  claim  follows  by  noting  L  (. x ,  •)  =  L  (A  (x) ,  •)  for  any  x  G  S.  □ 


The  following  elementary  inequality  can  be  proved  using  Legendre  transforms. 
Lemma  3.5.3.  For  a,q  G  [0,  oo)  we  have  al  (|)  +  a  (e  —  1)  >  q. 

We  now  study  the  asymptotic  behavior  of  the  proposed  local  rate  function  L. 

Proposition  3.5.4.  Given  a  >  0,  there  exist  constants  B  =  B  (a)  <  oo  and  C2  = 
C2  (B) ,  C3  =  C3  (B)  <  oo,  such  that 

(  C2||/3||log||/?||  if  x  eSa  and  (3  G  Ad_1,  ||/3||  >  B 
L(x,/3)  <  < 

C3  if  x  G  Sa  and  (3  G  Ad_1,  ||/3||  <  B. 

For  B  <  oo  sufficiently  large,  there  exists  c\  —  c\  (B)  >  0  such  that 

L{x,/3)  >  ci  ||/3||  log  ||/3|| 
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if  x  G  S  and  (3  G  Ad  1,\\/3\\  >  B.  In  particular,  L  (x,  /3)  is  super  linear  in  (3,  uniformly 
in  x. 

Proof.  Fix  a  >  0.  For  any  B  <  oo,  since  {(x,  (3)  G  5axAd_1  :  ||/3||  <  B }  is  a  compact 
subset  of  int(iS)  x  Ad_1,  the  uniform  boundedness  of  L  on  this  set  follows  directly 
from  Lemma  3.5.2. 

For  the  upper  bound  when  ||/3||  >  B,  we  first  assume  ||/3||  =  1.  By  Proposition 
3.5.1,  there  exists  a  bounded  vector  q  =  q  (/ 3 )  G  [0,  oo)lvl,  such  that  ^„gV+  vqv  =  (3 , 
and  qv  =  0  for  v  G  V\V+.  By  taking  an  open  cover  on  {/?  :  (3  G  Ad_1,  ||/3||  =  l},  one 
can  assume  max„j||(g||=i  \qv((3)\  is  bounded.  By  scaling,  it  follows  that  there  exists 
some  constant  Co  >  0,  such  that  for  any  (3  G  A6*'1,  there  exists  a  vector  q  G  [0,  oo) 
such  that  £„ev+  vqv  =  (3 ,  max,.  \qv\  <  Co  ||/3||,  and  qv  —  0  for  v  G  V\V+.  It  follows 
that  for  some  C4  <  00, 

L  (x,  (3)  <  c4  ^  qv  log  yt-t  <  C2 1|/3||  log  \\p\\ 

'V?  IX) 

t;GV 

if  ||/3||  A  B,  for  some  B  sufficiently  large  and  x  G  Sa.  This  finishes  the  proof  of  the 
upper  bound. 

Now,  consider  the  lower  bound  in  {(x,{3)  :  ||/3||  >  B,x  G  5}.  Using  Proposition 
3.4.3,  we  have  for  t  >  0,  a  —  tA,  and  M  <  00  defined  as  in  (2.3.1), 

L  (x,  (3)  >  (a,  (3)  —  H  (x,  a) 

>t  ii/3ii  -E  (x)  exp  (a,  v) 

vev 

>  t  \\p\\  -  M  |V|  exp  ( max  ||v||  t 
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Setting  t  =  max^y|H|  log  ||/3 1|,  this  implies 

£  (*,  P)  > - II/3II  log  m\  -  M  |V|  |(/3||  >  d  ||/3||  log  ||/3|| , 

max„ev  ||u|| 

for  some  constant  c±  >  0,  provided  ||/3||  is  sufficiently  large.  □ 


When  applying  Proposition  3.5.4,  we  will  always  assume  (without  mentioning 
in  explicitly)  that  B  is  chosen  to  be  a  sufficiently  large  number  so  that  all  parts  of 
Proposition  3.5.4  hold. 


Proposition  3.5.5.  Given  0  <  a  <  b  <  1  and  £  >  0,  suppose  that  7  G  AC  ([a,  b]  :  S^) 
satisfies  J8  L  (7  (s) ,  7  (s))  ds  <  00.  Let  {7<5}(5>0  G  D  ([a,  b\  :  be  such  that 

suPte[a,fe]  \\lS  (t)  -7(0||  -»■  0  as  5  — >  0.  Then  for  any  £  >  0,  there  exists  ho  = 
ho  (£,  e)  >  0,  such  that  for  5  <  ho?  J ^  L  (7  (s) ,  7  (s))  ds  —  J ^  L  (7^  (s) ,  7  (s))  ds\  <  e. 


Proof.  Fix  £  >  0,  0  <  a  <  6  <  1.  Let  A0  C  [a,  b]  be  the  set  for  which  7  (•)  is 
well  defined,  and  let  B  —  B  (£/2)  be  the  sufficiently  large  constant  from  Proposition 
3.5.4.  Then  [a,  6]  \_A0  has  measure  0.  Define  A  =  {s  G  A0  :  ||7(s)||  <  B}.  Given 
£■  >  0,  choose  B  larger  if  necessary  such  that 


Lie  (s)  ,j(s))ds  <  e. 


(3.5.5) 


J  [a,6]\^4 

Then  by  dominated  convergence  and  the  continuity  of  L(-,/3)  for  fixed  /3  G  A d_1 
established  in  Lemma  3.5.2,  we  have 


L(l8 


i7(« 


))  ds 


L  (7  (s)  ,f(s))ds. 


(3.5.6) 


Then,  with  constants  C2  =  C2  (B) ,  c\  =  C\  ( B )  from  Proposition  3.5.4,  it  follows 
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that 


[a,b]\A 


L  {A  (s)  >  7  (s))  ds  <  C2  /  ||t  (s)||  log  ||7  (s)||  ds 


[a,6]\A 


<  C2  /  C\  j 

<  C2e/ci. 


a,b]\A 


L  (7  (s)  ,7(s))ds 


Together  with  (3.5.5)  and  (3.5.6),  this  implies  that  for  <5  sufficiently  small, 


L('r(s),Af(s))ds-  /  L  (js  (s)  ,7(5))  ds 


<  £  +  £  +  C2£ / Ci  —  (2  +  C2j C\)E. 


□ 

Lemma  3.5.6.  Suppose  that  7  G  AC  ([0, 1]  :  5)  satisfies  j ^  L  (7  (s) ,  7  (s))  ds  <  00 
for  some  0  <  a  <  b  <  1.  27ien 


117  (*)  —  7  (a)ll  log  ■ 


0  as  t  {  a. 


Proof.  Fix  t  G  (a,  5)  and  let  A0  C  [a,  6]  be  the  set  for  which  d(-)  is  well  defined. 
Then  [a,  b] \A0  has  measure  0.  Now  observe  that  ||7(-)||  l°g  |[t  (*) II  e  L1  ([a,  6]  :  5). 
Take  some  sufficiently  large  B  <  00  from  Proposition  3.5.4,  and  let 
A  =  {s  G  [a,  b]  :  ||y  (s)||  <  B}.  Then  by  Proposition  3.5.4,  there  exists  ci  =  Ci  ( B ), 
such  that 


||7(s)||  log  H7  (s)||  ds  <  — 


C1  J[a,b]\A 


L  (7  (s) ,  7  (s))  ds  +  /  B  log  Bds 


< 


Cl 


L  (7  (s) ,  7  (s))  ds  +  (. B  log  B )  ( b  -  a ) 
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By  Jensen’s  inequality, 


Il7(s)ll  lQg  Il7(s)ll  ds  >  (t~a) 


7  (t)  ~  7  (a) 

log 

7  (t)  ~  7  («) 

t  —  a 

t  —  a 

=  117  (t)  —  7  (a)ll  log 


117  (t)  —  7  (a)ll 


t  —  a 


The  lemma  follows  by  observing  that  both  the  left  hand  side  and 

||7  (t)  -  7  (a) ||  log  H7  (t)  -  7  (a) ||  goes  to  zero  as  t  l  a.  □ 

Lemma  3.5.7.  Let  cq  (<5)  be  given  such  that  cq  (6)  — >  1  as  5  — >  0.  Suppose  that 
x  E  S,  {a:<5}(5>0  Eint(S),  are  such  that  ||a;  —  x<5||  — >  0  as  6  — >  0,  and  for  any  5  >  0 
and  v  E  V,  <  cq  (8).  Then  there  exists  c  =  c(5)  that  only  depends  on  Co  (J) 

arid  llx  —  x<5|L  and  goes  to  zero  as  5  ^  0,  such  that  for  any  (3  E  C  {v  :  v  E  Vx}, 


L  (x,  (3)  <  (1  +  c  (<5))  L  (re*5, 0)  +  c  (J) . 


Proof.  Fix  (3  E  C  {v  :  v  E  14},  and  take  q  E  [0,  00)  such  that  ^2veVx  vqv  =  (3  and 
qv  =  0  for  v  E  V\VX.  By  the  continuity  of  {A„  (•)}  and  the  fact  that  they  are  either 
identically  zero  or  bounded  uniformly  from  below  in  int(«S)  (Condition  3.1.1),  it 
follows  that  for  x  eint(<S)  and  S  sufficiently  small,  Vxs  =  Vx.  For  x  E  OS,  Vx  C  Vxs 
if  {^<5}(5>0  approximate  x  from  the  interior.  Therefore  for  5  sufficiently  small,  the 
same  vector  q  we  take  before  satisfies  s  vqv  =  (3  and  qv  —  0  for  v  E  V\VX5.  It 

suffices  to  show  that  for  any  <5  >  0, 


y:  (x)  i 

vevx 


<{l  +  c(S))J2Xv  {x5)t 
vEVx 


+  C  (J)  , 


(3.5.7) 


where  c(<5)  is  independent  of  q. 
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Note  that  there  exist  C\  <  oo,  such  that 


Y  K  {x)  i 


vevx 


\v  (x) 


Y Xv  (x*) 


vevx 


E  Xv  (x*5)  v-^  / . 

«"log  MU  +^(A‘ 


USVa: 


uc 


A„  (a;5)  < 

-  A„  (a;5)) 


<  log  c0  (5)  Yqv  +  c: 

vevx 

<  log  c0  (S)  Y  (^v  (x5) 

veVx  X 

<  log  Co  (5)  Y  Xv  (x<5)  ^ 


v£Vx 


l\\X  —  X 


q 


v£Vx 


Xv  (x 


^  ( x ( e  —  1)^  +  C\ 

Ssi+C2{8), 


Xv  ( Xs 

qv 


\x  —  x 


for  C'2  (5)  =  log  Co  (<f)  M  |  V |  (e  —  1)  +  C\  ||a:  —  a;5 1 1 ,  where  the  first  inequality  follows 
from  the  assumption  and  Lipschitz  continuity  of  {A„  (•)},  and  the  second  inequality 
follows  from  Lemma  3.5.3  by  taking  a  =  Xv  ( xs )  and  q  =  qv.  It  suffices  to  take 
c  (<f)  =  max  {log  c0  (<f) ,  C2  (<f )}-  □ 


Notice  that  for  (3  £  C  {v  :  v  G  Vx}t  the  conclusion  of  the  above  Lemma  holds 
trivially  since  the  right  hand  side  is  infinity. 

For  t  <G  [0, 1]  and  c  >  0,  define 

7c  (s)  =  7  (cs) ,  s  G  [0,  t]  , 

which  is  a  time  reparametrization  of  7.  The  next  result  is  used  in  the  proof  of  the 
locally  uniform  LDP  in  Section  3.7.  It  states  that  given  a  path  7  with  finite  cost, 
the  cost  of  the  path  depends  continuously  on  the  reparametrization  of  time. 

Proposition  3.5.8.  Fort  G  [0,1)  given,  suppose  7  G  AC  ([0,1]  :  S)  is  such  that 
It  (7)  <  00.  Then  the  function  c  1— »  It/C  (%)  is  continuous  at  1. 
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Proof.  First  note  that  for  c  close  to  1,  7C  G  AC  ([0,  t/c]  :  S )  and 

r-t/c  rt. 

h/c{lc)=  /  L(-/(cs)  ,cy(cs))ds  =  -  /  L  (7  (r ) ,  07  (r ))  dr . 

Jo  c  J o 

It  suffices  to  bound  the  integral  of  ^(7,07)  —  L( 7,7).  Recall  the  definition  of  L 
in  (3.1.3).  Since  7  is  absolutely  continuous,  7  (w)  is  a.s.  well  defined.  For  any  fixed 
u  G  [0,  t]  such  that  7  ( u )  is  well  defined,  for  any  e  >  0,  there  exists  q  G  [0,  00)  ,  such 

that  )T^ev  V(P  =  c7  (M)  and 


(7  (u))£ 

v£V 


f  qv/c  \ 

(7  (u))J 


<  L(7  («),  7  («))+£. 


We  also  have 


vev 


J2\Tloe 


qv/c 


C  Xy  (7  (■ U ))  C 


-  —  +  A„  (7  («)) 


v&V 


+  1 


lo§ 


K  (7  (w)) 


(7  (u)) 


+  K  (7  («))  )  +  (  “  log  “  )  qv 


uev 


veV 

1 

>  —L  (7  («)  ,c7(m))  + 
c 


-log- 

c  c 


vev 


+  ( 1  -  - )  Xv  (7  (m))  • 


vev 


Thus 


—L  (7  (u) ,  cry  (u))  —  T  (7  («),7(«))  <e-  log  — ^  ^2qv~  fl  -  A„  (7  («))  • 

c  Vc  c/  veV  \  CJ  veV 

(3.5.8) 
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Similarly,  by  taking  q  G  [0,  oo) lvl ,  such  that  vqv  =  7  (u)  and 


(7  {u))£ 

v£V 


f  cqv  A 

(7  («))/ 


<  L  (7  (u) ,  07  (u))  +  ce, 


an  analogous  computation  yields 


~L  (7  (u) ,  cy  (u))  —  L  (7  (u) ,  7  (u))  >  (log  c)^qv-  A--")  ^  (7  («))-£. 

vev  '  ' 

(3.5.9) 


vGV 


We  apply  Lemma  3.5.3  with  a  =  A„  (7  (u))  and  q  =  qv/c,  and  the  boundedness 
of  Xv  to  obtain 


1  \  - 

-  V  (lv  < 
c 

vev 

< 


( Ati 

ueV  7 


f  Qv/c  \ 

(7  (u))J 


L  (7  («),7(«))+e  +  ci 


+  (7  («))  (e  -  1) 


for  some  c\  <  00.  Thus,  combining  with  (3.5.8)  and  (3.5.9),  we  see  that  for 
c  sufficiently  close  to  1, 


-L  (7  (u) ,  C7  («))  -  L  (7  («) ,  7  (u)) 


<  M  |V| 


1 

1  -  - 

c 


+  max  log  -,  clog  c\(L(  7  (w) ,  7  (w))  +  £  +  ci)  +  e. 


Since  /  (7)  is  finite,  one  can  integrate  over  [0,  t],  take  c  — >  1,  and  then  send  £  — >  0  to 
complete  the  proof.  □ 


3.6  Proof  of  the  LDP  lower  bound 
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We  now  turn  to  the  proof  of  the  LDP  lower  bound.  It  suffices  to  show  that  for  any 
fixed  trajectory  7  e  D  ([0, 1]  :  S),  given  any  £  >  0  and  <5  >  0  there  exists  rj  >  0  such 
that  if  \\pn  (0)  —  7  (0)||  <  rj  for  all  n  large  enough, 

liminf-logPdl//1  -  7IL  <  S)  >  -I  (7)  -  e. 

n— KX5  n 

Without  loss  of  generality  we  assume  /  (7)  <  00. 

One  source  of  difficulty  here  is  that  the  transition  rates  of  fin  may  tend  to  zero 
as  /in  approaches  the  boundary  of  S,  which  will  lead  to  singularity  of  the  local  rate 
function.  Our  approach  here  adapts  an  idea  from  the  study  of  a  discrete  time  model 
in  [15].  We  first  show  that  the  singularity  can  be  avoided  except  for  t  —  0,  by  slightly 
perturbing  the  original  path,  with  arbitrarily  small  additional  cost. 

3.6.1  Perturbation  argument 

We  start  with  a  direct  evaluation  of  the  hitting  probability  of  jump  Markov  processes 
on  a  finite  state  space. 

Lemma  3.6.1.  Let  {Y  (£)}t>0  be  a  jump  Markov  process  with  finite  state  space 
(so,  si, ...,  sn}-  For  i  =  0, ...,  n  —  1,  suppose  that  the  jump  rate  from  state  Si  to  Sj+i 
is  bt+i ,  and  the  sum  of  jump  rates  frorn  state  Si  to  all  other  states  is  bounded  above 
by  c  <  00.  IfY  (0)  =  so,  then 

P  (Y  (; t )  =  sn)  >  (IIJLi&i)  tn  exp  (— ct)  . 

n! 

Proof.  Let  p  (t)  be  the  probability  distribution  of  the  process  at  time  t :  pi  (t)  = 
P(y  (t)  =  Si).  Then  the  Kolmogorov  forward  equation  takes  the  form  p  =  Ap, 
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where  A  is  the  rate  matrix  for  Y.  By  the  comparison  principle  for  ODEs,  it  follows 
that  pi  ( t )  >  rt  ( t )  for  all  i  G  X,  where  r  (t)  is  the  solution  to 

' 

r0  =  -cr0, 

<  fi  =  biTi-i  —  c?y,  i  =  1,  ...,n, 

r(0)  =  eS0. 

Solving  this  equation  explicitly  gives  pn  (t)  >  rn  (t)  =  A  (fl”=16j)  tn  exp  (—ct) .  □ 

The  idea  of  the  perturbation  argument  is  as  follows.  Recall  the  definition  of  Sa 
in  (3.5.2).  For  any  a  >  0  fixed,  by  Remark  3.1.2,  the  rates  A„  (•)  are  either  identically 
zero  or  uniformly  bounded  below  away  from  zero  within  Sa.  Therefore,  a  standard 
approximation  argument  can  be  used  to  establish  the  LDP  in  Sa ,  uniformly  with 
respect  to  the  initial  condition.  When  7  (0)  =  x  G  S/Sa,  by  using  Condition  3.1.3, 
one  can  construct  a  perturbed  trajectory  of  7,  that  hits  Sa  in  an  arbitrarily  short 
time  as  a  —>  0,  and  in  such  a  way  that  the  difference  in  cost  between  7  and  the 
perturbed  trajectory  can  be  made  sufficiently  small. 

Lemma  3.6.2.  Assume  the  family  of  jump  rates  {A„  (x) ,  x  G  S,v  G  V}  satisfies 
Conditions  2.3.1,  3.1.1  and  3.1.3.  Consider  7  G  AC  ([0, 1]  :  S)  such  that  I  (7)  <  00. 
Then  given  any  e  >  0,  there  exists  b  >  0,  D  <  00  and  a  trajectory  v  G  AC  ([0, 1]  :  S ) 
such  that 

i)  v  (0)  =  7  (0)  and  ||u  -  7^  < 

ii)  Vj  (■ t )  >  btD  for  j  =  1, ...,  d  and  any  t  G  [0, 1], 
in)  I  (v)  <  I  (7)  +  e. 
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Proof.  For  0  <  p  <  1  define  vp  =  pp  +  (1  —  p)  7,  where  p  is  the  law  of  large 
numbers  trajectory  defined  in  (2.3.2)  with  p  (0)  =7(0).  Let  Cd  be  the  diameter  of 
S.  Then  we  have  \\vp  —  7^  =  p  \\p  —  7^  <  Cdp.  Also,  one  has  the  lower  bound 
Vj  ( t )  >  ppj  ( t )  >  pbtD  for  some  D  <  00  by  Condition  3.1.3.  Therefore,  it  suffices 
to  show  iii).  We  first  show  that  there  exists  c(p)  <  00  which  goes  to  zero  as  p  — >  0, 
such  that  for  any  t  e  [0, 1], 

L  (■ vp  (t) ,  7  (t))  <  (1  +  c  (p))  L  (7  (t) ,  7  (t))  +  c  (p) .  (3.6.1) 

Indeed,  since  7 i(t)/vf(t)  <  and  \\vf  (t)  —  7^  (f)||  <  CdP  for  i  =  1  by 

Condition  3.1.1 

XWW)  -  (ivy)  exp  (/  (p))  • for  every  ” 6  v- 

where  /  (p)  =  maxse[0jcdp]  /o  (s)-  Since  the  right  hand  side  goes  to  1  as  p  — >  0,  (3.6.1) 
follows  by  applying  Lemma  3.5.7  with  x  —  7  (t )  and  xp  =  vp  ( t ). 

Likewise,  pi  ( t )  /vf  ( t )  <  1/p  and  \\vf  ( t )  —  pi  (t)||  <  Cd  (1  —  p)  for  i  —  1, ...,  d, 
thus  Condition  3.1.1  implies 

exp(/(1_p))'  (3'6'2) 

Therefore,  by  the  definition  of  L  (3.1.3)  and  the  fact  that  p{t)  =  v\v  (p  (t)), 
we  have 


L{vp  (t),  p(t)) 


< 

< 


v£V 


(  id  (t))  \ 

VA„  (vp  (t))J 


Cr2  log  1/p  +  C2/  (1  —  p)  +  C'2 


for  some  C2  <  00,  where  we  apply  (3.6.2)  for  the  last  inequality. 
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By  (3.6.1)  and  the  convexity  and  nonnegativity  of  L  (x,  •)  stated  in  Proposition 
3.5.2,  one  has 

L  (• vp  (; t ) ,  vp  (t))  <  L  (vp  (; t ) ,  7  (t))  +  pL  (■ vp  (; t ) ,  p  (t)) 

<  (1  +  c(p))  L  (7  (t) ,  7  (t))  +  c5  (p) . 

for  C5  (p)  =  C2P  log  1/p  +  Cspf  (1  —  p)  +  C4P  +  c  (p).  Integrating  both  sides  of  the 
last  inequality  over  [0, 1],  we  get 

I  (■ vp )  <  (1  +  C  (p))  /  (7)  +  c5  (p)  , 

and  thus  iii)  holds  with  v  =  vp  for  p  >  0  sufficiently  small.  □ 

In  view  of  Lemma  3.6.2,  it  suffices  to  establish  the  lower  bound  for  paths  7  G 
AC  ([0, 1]  :  S )  with  I  (7)  <  00  that  satisfy  the  additional  condition 

7 i(t)  >  b0tD  for  some  60  >  0,  D  <  00  and  for  all  t  G  [0, 1].  (3.6.3) 

3.6.2  Analysis  for  t  £  [0,  r] 

Given  h  >  0,  for  t  >  0  sufficiently  small  we  can  use  excursion  bounds  for  jump 
Markov  processes  (Lemma  3.6.4  below)  to  establish  a  lower  bound  for  the  quantity 

P  f  sup  ||pn(t)  -7(t)||  <  5  )  . 

\te[o,r]  J 

The  more  difficult  part  is  to  obtain  for  any  0  <  a  <  5  a  lower  bound  for 
P(llAt”('r)  —  7(r)ll  <  that  is  uniform  in  p”(0)  as  long  as  ||pn(0)  —  7(0) ||  is  suffi- 
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ciently  small. 

Given  £  >  0,  r  e  (0, 1],  for  any  a  >  0,  define  the  penalty  function  g  :  S  — >  M 


g(x) 


0  I \x  —  7(r)l  I  <  ui 
2e  else. 


(3.6.4) 


We  then  have 

p(ll/A0  -  7(011  <  <0  +  e~2ne  >  E  [exp(— ng(/in(r)))] .  (3.6.5) 


Recall  from  Theorem  3.3.6 


1 

n 


log  E  [exp (~ng(pn  (r)))] 


inf  E 


s  eA 


®|V| 


E/  xv(fin(t))£ 


.uev 


(  MO  \ 

VA  Z(p,n(t))J 


dt  +  g(pn  (t))  :  pP 


A”  (d,  /in(0)) 


We  now  state  the  main  result  of  this  subsection. 

Lemma  3.6.3.  Assume  the  sequence  of  deterministic  initial  conditions  {gn  (0)}ngN 
converges  to  p®  E  S  as  n  tends  to  infinity.  Then,  given  e  >  0  and  5  >  0,  there  exists 
t  >  0  such  that  for  any  a  >  0,  there  exists  g  =  g  (a)  >  0,  such  that  ||p0  —  7  (0)  ||  <  g 
implies 

lim inf  —  log P  f  \\pn  (r)  —  7  (r) ||  <  a,  sup  ||/7n(s)  -  7  (s)||  <  s)  > 

Proof.  The  idea  is  to  argue  that  for  large  n  and  small  r,  pn  stays  close  to  a  com¬ 
municating  path  (defined  in  Condition  3.1.4)  that  connects  7(0)  to  y(r).  Since 
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the  jump  rate  is  bounded  below  away  from  zero  along  such  a  path,  one  obtains  a 
nice  upper  bound  for  the  cost.  By  Condition  3.1.4  and  Remark  3.1.6,  there  exists 
a  communicating  path  ft  G  C  ([0,  r]  :  S),  with  ft  (0)  =7  (0)  and  (ft  (t)  =7  (r),  and 
F,U  <00,  {vm}Fm=\  C  V  and  0  =  to  <  <  •  •  •  <  =  t,  such  that 


av(t)v,  a.e.  t  G  [0,  r] , 


vev 


where 


av(t)  = 


U1[tm-utm)  (t)  if  v  =vm,m  =  1 


0  v  i-  K)L  1  • 

Also,  by  Condition  3.1.4  and  (3.6.3),  there  exist  d,p,  D  <  00  and  cq,  c  >  0,  such  that 


I  Moo  =  ~ 


ft  (t) 


dt  <  d 


/ 1|7  (T)  ”  7  (0)|| 


r 


(3.6.6) 


and 


A„m  {ft  (s))  >  c  ^min7j  (r)j  >  c0rDp,  if  s  G  [tm-i,  tm\ ,  m=l,...,F.  (3.6.7) 

Define  fin  =  An  (a,  fin  (0)),  where  A"  is  as  defined  in  (3.3.9).  The  LLN  for  Poisson 
random  measures  implies  that  {/fn}neN  converges  uniformly  on  [0,r]  in  probability 
to  /i,  where  fi  (0)  =  po, 

— p(t)  =  yj  av  ( t)v ,  a.e.  t  G  [0,  r] .  (3.6.8) 


By  the  Lipschitz  continuity  of  (•),  the  fact  that  \\p  (s)  —  ft  (s)||  =  ||po  —  7  (0)|| 
since  they  use  the  same  velocity,  and  (3.6.7),  for  any  fixed  r,  there  exists  some 
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r/o  (r)  >  0,  such  that  for  any  rj  <  rjo  (r),  if  ||po  —  7  (0)||  <  rj,  then 


Km  (Ms))  >  7pTDp,  for  s  e  ,  m  =  1,  ...,F. 


(3.6.9) 


For  r/  <  r/o  (r),  we  can  bound  the  cost  along  the  path  /i  by  (each  inequality  will  be 
explained  below) 


dt 


v£V 

<  |  V  |  r 


j  IIt  (t)  —  7  (0)11 


r 


log  Il7  (t)  —  7  (0)  ||  -  log 


£0  Dp+l 

o  ' 


M 


<  C4||7(t)  —  7  (0)  ||  log  ||7  (r )  -7(0)11  +  C±\\i{t)^  7  (0)||  log  -  +  C4t 


(3.6.10) 


for  some  C4  <  00,  where  M  is  defined  in  (2.3.1).  Here  we  used  the  definition  of  L 
and  (3.6.8)  to  obtain  the  first  inequality;  (3.6.6),  (3.6.9),  and  (2.3.1)  for  the  second 
inequality.  By  Lemma  3.5.6,  ||q  (r)  —  7  (0)  ||  logi  — >  0  as  r  — >  0,  together  with  the 
continuity  of  7  we  obtain  IT  ( Jx )  — >  0. 


We  next  bound  the  costs  for  the  jump  processes  {/in}neN  by  making  use  of  its  law 
of  large  numbers  limit.  For  fixed  r,  we  use  the  fact  that  are  Lipschitz  continuous 
(by  Condition  2.3.1),  and  Km(p>(t))  is  uniformly  bounded  below  away  from  zero  if 
||po  -  7  (0)11  <  Vo  (r)  (by  (3.6.9)).  Therefore,  A”  ( fxn  (t))  £  {av/ A”  (, fin  (t)))  converges 
in  probability  to  A„  (p.  (t))  £  (av/\v  (p  (t)))  uniformly  for  t  G  [0,  r] .  Then  for  rj  < 
min  {770  (r) ,  cr/2},  applying  the  dominated  convergence  theorem  and  using  the  upper 
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semicontinuity  of  g  defined  in  (3.6.4),  we  have 

jT K mm e (A r^l)))  dt + IW <T»  :  Pn  =  A” («. 

<  C4II7  M  -7(0)11  log  N'T  (t)  -7(0)11  +Ci  ||7  (t)  -7  (0)11  log  -  +C„T, 

r 

where  the  last  inequality  follows  from  (3.6.10)  and  (3.6.4).  Choose  r  >  0  sufficiently 
small  such  that  the  last  expression  is  less  than  e/2.  Combining  the  last  display 
with  the  representation  formula,  for  all  sufficiently  large  n  and  sufficiently  small  77, 
1 1  p0  -  7(0)||  <  V  implies 

—  —  logE  [exp(— np(p”(r)))]  <  e/2.  (3.6.11) 

n 

When  combined  with  (3.6.5),  this  gives  a  lower  bound  on  P(||/in(r)  —  7(t)||  <  a). 

We  will  conclude  the  argument  by  establishing  an  upper  bound  for  the  excur¬ 
sion  probability  for  /in  during  [0,  r].  Given  e  >  0,  applying  a  standard  martingale 
inequality  (stated  as  Lemma  3.6.4  below),  for  sufficiently  small  r  we  have 

P  (  sup  ||p”(s)  —  p”(0)||  >  -  ]  <  2dexp  (— ne) . 

\0<s<r  3 ) 

On  the  other  hand,  by  taking  r  smaller  if  necessary  we  can  guarantee  that 
suPse[o,r]  Il7  (s)  —  7  (0)||  <  5/3.  It  follows  that  for  r\  G  [0,  |] , 

P  (  sup  ||/in(s)  -  7  (s)||  >  s)  <  P  f  sup  ||/in(s)  -  p”(0)||  > 

\0 <S<T  J  \0<S<T  o  ) 

<  2d  exp  (—ne) 
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Combining  this  with  estimates  (3.6.5)  and  (3.6.11)  we  arrive  at  the  desired  conclu¬ 
sion.  □ 

The  following  lemma  is  an  adaptation  of  Lemma  2.3  in  [13].  The  lemma  follows 
from  bounds  for  certain  exponential  martingales. 

Lemma  3.6.4.  Let  C\  =  max„ev  ||v|| ,  C2  =  M  |V|  C\ ,  and  define 

Hbi=-fb(l0g(b/C2)-l) 

V-l 

for  b  >  C2.  Then  £(b)  /b  00  as  b  00,  and  given  any  5  >  0,  for  all  r  <  2^c 

P  (  sup  || nn  ( t )  —  yn  (0)||  >  S 1  <  2dexp  (—ml  ( — 

\o  <t<r  )  \  \2  Vdr 


3.6.3  Analysis  for  t  £  [r,  1] 

Let  B  (x,  r)  denote  the  open  Euclidean  ball  centered  at  x  with  radius  r.  Also,  for 
7  G  AC  ([t,  1]  :  S )  such  that  7  (r)  =  y,  we  denote 

Iy  (7)  =  /  L(l(s),i(s))ds, 

to  emphasize  the  dependence  on  y.  Pyn  and  Ey„  denote  the  probability  and  expecta¬ 
tion,  respectively,  conditioned  on  jin  (r)  =  yn.  Dehne  the  mapping  A”  :  x5-> 

D([r,l]:S)  by 

K  («,P)  (*)  =  P  +  J(v  f  [  ![o ,av(s)](x)-K(dsdx), 
tigv  Tta  Jy  n 
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for  a  E  and  p  E  S.  It  suffices  to  study  a  path  7  that  satisfies  (3.6.3).  Therefore 
we  can  assume  Iy( 7)  <  00,  and  that  there  exists  f  >  0  such  that  7  ( t )  lies  in  S ^  for 
t  E  [r,  1] .  We  will  prove  the  following  uniform  Laplace  principle  upper  bound  for 

on  M]; 

Proposition  3.6.5.  Let  f  >  0  as  defined  in  the  previous  paragraph,  and  fix  y  E 
S^.  Then  there  exists  a  >  0  such  that  for  any  bounded  and  Lipschitz  continuous 
functional  F  on  d  ([r,  1]  :  S), 

liminf  inf  ( -  logE^  [exp (-ruF (//*))]  —  G  (yn,  F)  )  >  0,  (3.6.12) 

n— >00  yn&B(y,o)  \n  J 

where 

G  (y,  F)  —  —  inf  [P(7)  +  F( 7)].  (3.6.13) 

7eAC([r,l]:5«) 

In  particular,  this  implies  the  uniform  large  deviation  lower  bound:  for  any  e  >  0 
and  S  >  0,  there  exists  a  >  0  such  that  for  any  yn  E  B  (7  (r) ,  a), 

liminf- log Fyn  (  sup  \\pn  (t)  -  7  (t) ||  <  5  )  >  -J7(r)  (7)  -  % 

n^°°  n  ye[r,i]  J  2 

The  proof  of  Proposition  3.6.5  relies  on  the  following  approximation  argument, 
which  we  now  establish. 

Fix  y  E  and  a  bounded  and  Lipschitz  continuous  functional  F  on  D  ([r,  1]  :  S). 
By  Proposition  1.2.7  of  [12],  to  prove  (3.6.12),  it  suffices  to  show  that  for  any  sequence 
{yn}neN  such  that  \\yn  -  y\\  0  as  n  ->  00, 

lim  inf  —  logE.y„  [exp(— nF(pn))]  >  G  (y,  F) . 

n— kx)  77, 


(3.6.14) 
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In  other  words,  it  suffices  to  show  that  for  any  £  >  0  and  G  AC  ([r,  1]  :  S 
such  that  -  +  F( %))  >  G  (y,  F )  -  e, 


lim  inf  —  log E,y„  [exp(-nF  (//"))]  >  -  (Iy( %)  +  F( %)) , 

n—>  oo  77, 


or 

lim  sup--  log  Eyn  [exp(-nf(/tn))]  <  Iy(  %)  +  F(  %). 

n—>  oo  Tl 

Fix  £■  >  0  and  denote  simply  by  7.  We  now  approximate  7  by  a  piecewise 
linear  path.  Let  A  =  Pz  for  some  m  G  N.  For  k  =  0,  —  1  let  aA  = 

i  /r+fcA+1)A  7  («)  ds.  Dehne 

7A  (t)  =  aA  if  t  G  (t  +  kA,  t  +  (k  +  1)  A),  k  =  0, m  —  1, 

and 

7A  (t)  =  y  +  J  yA  (s)  ds  for  t  G  [r,  1] .  (3.6.15) 

Then  yA  is  the  piecewise  linear  interpolation  of  the  continuous  process  7  with  mesh 
size  A.  Note  that  for  any  v  G  V  and  t  G  [r,  1],  A„(7A  (t))  is  continuous  and  uniformly 
bounded  away  from  zero.  The  proof  of  (3.6.12)  thus  relies  on  the  following  standard 
approximation  result. 

Lemma  3.6.6.  Let  7  G  AC  ([r,  1]  :  <S^)7  and  define  7A  as  in  (3.6.15).  Then  for  any 
e  >  0,  there  exists  A0  (e)  >  0,  such  that  for  any  A  <  A0  (e),  and  a.e.  t  G  [r,  1], 
there  exists  a  piecewise  constant  vector  qA  (t)  such  that  Yhv&vV(lv'  ft)  =  7A  ft )?  and 

-yt<PM  +  ,  (3.6.16) 

Proof.  We  first  claim  liniA^o  Iy  (7A)  =  Iy  ftt)-  Dehne  G  [0,  00) such 
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that  for  v  G  V, 


xk,v 


1 

A 


r»T+(fc+l)  A 


*  r+kA 


\v  (7  0))  ds. 


Define  xA  (t)  —  xA  if  t  G  [r  +  /cA,  r  +  (/c  +  1)  A).  Recall  the  definition  of  L  given 
in  (3.5.4).  By  the  joint  convexity  of  L  established  in  Lemma  3.5.2  and  Jensen’s 
inequality, 


L  [xA  (s) ,  yA  (s))  ds 


< 


m—1 

A^2Lixhi  ak) 

k= 0 

m- 1  „r+(fc+i)A 

/  ^(A(7(s)),7(s))^ 

k= 0  Jr+kA 


Iy  (7) . 


Since  Iy  (7)  <  00  by  the  continuity  of  L  and  dominated  convergence  theorem  we 
conclude  that 

fim  j  L  (xA  (s) ,  yA  (s))  ds  =  Iy  (7) . 

Furthermore,  note  that  L  (A„  (7A  (s))  ,7A  (s))  =  L  (s)  ,jA  (s)),  and 
supserT  l]  ||  (jA  (s))  —  xA  (s) 1 1  — >  0  as  A  — >  0.  Apply  Lemma  3.5.7,  we  have 


< 


< 


c(A)sup  [  L  (xA  (s) ,  yA  (s))  ds  +  c  (A) 

A>0  JT 

c  (A)  Iy  (7)  +  c  (A) , 


for  some  c  (A)  — >  0  as  A  — >  0.  The  claim  is  proved  on  taking  A  — >  0. 

For  k  =  0, ...,  m  —  1,  fix  some  tk  G  (r  +  kA,  r  +  (k  +  1)  A).  We  can  take  qA  G 
[0,  cx))|v|  such  that  YjVev  Vcitv  —  ak ,  and 

L{ 7A  (h) ,  7A  (h))  >  Xv^A  (tfc))£  f  7  f  •  (3.6.17) 

Since  jA  G  AC  ([r,  1]  :  S^),  by  the  uniform  continuity  property  of  rate  function  in 
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the  state  variable,  which  is  stated  in  Proposition  3.5.5,  choosing  A  sufficiently  small 
we  have 


ra-l  f(k+l)A +T 

iy  hA)  l^a  (**)  >  (tk))dt 

k= 0  dkA+r 


£ 


(3.6.18) 


Dehne 


qA  (t)  =  Qk  (t) ,  if  t  e  (t  +  kA,  t  +  (k  +  1)  A) ,  k  —  0, ...,  m  —  1. 


Applying  Lemma  3.5.7  again  and  integrate  over  t  e  [r,  1],  there  exists  d  (A)  which 
goes  to  zero  as  A  — >  0,  such  that 


^(7A  (t)) 


'JT  uev 
m-1  />(fc+l)  A+r 

E  /  EA«(7  MX 

fc=0  JkA+T  V£V 

m-1  /.(fc+l)A+r 


dt 


(3.6.19) 


qL 


dt 


<  d 


lib - _L  n 

'“Si 


k= 0 JkA+T 


A„(7a  (tk)) 

EV7i(tO)<(jCf^S5jj<tt  +  rf(A)<3.6.20) 


By  (3.6.17),  (3.6.18),  and  the  fact  that  Iy  (yA)  is  finite,  one  can  take  A  smaller  if 
necessary  so  that  the  LHS  of  (3.6.20)  is  less  than  e/6.  The  conclusion  follows  by 
combining  (3.6.17),  (3.6.18),  (3.6.20),  and  the  convergence  of  Iy  (yA)  to  Iy  (7).  □ 


We  now  complete  the  proof  of  Proposition  3.6.5.  By  Lemma  3.6.6,  for  any  e  >  0, 
there  exists  A  sufficiently  small  and  a  collection  of  piecewise  constant  functions 
{gA  (•)},(, ev  011  tr’  1]  ff1&f  satisfy  (3.6.16).  It  follows  directly  from  the  LLN  for  Poisson 
random  measures  that  as  n  — >  00,  fin  =  A/  ( qA,yn )  converges  uniformly  on  [r,  1]  in 
probability  to  yA.  Therefore,  by  the  uniform  continuity  of  A,,  (•)  i  ((/A/A„  (•))  on  S* 
and  the  uniform  convergence  of  A”  (•)  to  A„  (•)  by  Condition  2.3.1, 

A”  (/!”('))  ^  (<?A  (')  /K  (/!”(')))  converges  uniformly  on  [r,  1]  in  probability  to 
\,(7A  (■))£  (gA  (•)  /A„(7A  (•))) .  Combining  the  variational  representation  formula 
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(Theorem  3.3.6),  (3.6.16),  and  the  dominated  convergence  theorem,  for  any  Lipschitz 
continuous  functional  F  on  D  ([r,  1]  :  S),  we  have 


limsup - logEyri  [exp (—nF  (//*))] 

n— >00  Tl 


=  lim  sup  inf  E. 

n->oo  aeAf^ 


y-n 


<  lim  sup  E 


E„ev  Jr  K  (/*"(*))  l  (^%)j  dt  +  F{JF)  : 
Jln  —  A”  (a,  yn) 

Qv{t) 


v&> 


=  E» 


dt  +  F(jln):fin  =  A?(qA,yn) 


<  Ey  |U  (7)  +  e  +  ^(7A)] 

<  Iy  (7)  +  F  (7)  + 


for  all  A  sufficiently  small,  where  the  last  inequality  follows  from  the  continuity  of  F, 
and  the  fact  that  suptgrTjli  1 1 7^  (t)  —  7  (t)\\  — >  0  as  A  — >  0.  By  the  Markov  property, 
the  LDP  lower  bound  follows  from  Lemma  3.6.3  and  Proposition  3.6.5. 


3.7  The  Locally  Uniform  LDP 


We  now  turn  to  the  proof  of  Theorem  3.1.15.  Fix  t  G  [0, 1].  As  shown  in  Corollary 
3.1.11,  one  can  express  the  rate  function  Jt  of  {/in  (t)}ngN  in  terms  of  a  variational 
problem.  In  what  follows,  fix  x  G  S  and  such  that  xn  G  Sn  and  \\xn  —  x||  — >  0 

as  n  — >  00. 
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3.7.1  Proof  of  the  upper  bound 

Given  any  £  >  0,  recall  that  B  (x,e)  denote  the  Euclidean  ball  centered  at  x  with 
radius  e.  For  n  sufficiently  large  such  that  xn  G  B  (x,e),  by  the  LDP  upper  bound 
stated  in  Corollary  3.1.11, 

X  x 

lim  sup  —  log  P  (/in  (t)  —  xn)  <  lim  sup  —  log P  (fin(t)  G  B  (x,e)) 

n— >oo  Tl  n— >oo  Tl 

<  -J£t(y0,x), 


where  we  define 

—  inf  {/*  (7)  :  7  G  D  ([0, 1]  :S),7(0)  =  /xo,7(t)  eB(x,e)}.  (3.7.1) 


To  prove  the  upper  bound,  it  suffices  to  show  that 

lim  inf  J (/r0,  x)  >  Jt  (/Jo,  x) .  (3.7.2) 

£ — >0 

Lemma  3.7.1.  Assume  Condition  3. 1.1 4.1)  holds.  Then  there  exists  a  function 
c  :  [0,  cx))  — >  [0,  ex)  such  that 

i) .  c  (s)  — >  0  as  e  — >  0,  and 

ii) .  given  any  e  >  0  and  x,y  G  S  such  that  ||a;  —  y\\  <  e,  one  can  construct  a 
path  7  G  AC  ([0,  e\  :  S)  such  that  7  (0)  =  x,  7  (e)  =  y,  and  J£  (x,  y)  <  I£  (7)  <  c(e), 
where  I£,J£  are  defined  in  (3.1.5),  (3.1.9),  respectively. 

Before  proving  Lemma  3.7.1,  we  first  describe  how  it  can  be  used  to  prove  Lemma 


3.1.16  and  (3.7.2). 
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Proof  of  Lemma  3.1.16.  For  any  £  >  0,  take  t  >  0  and  7  G  AC  ([0,  t]  :  S)  such  that 
7(0)  =  x,  'jit)  =  y,  and  It( 7)  <  V  (. x,y )  +  ej2.  Given  5  >  0,  and  any  ys  G  S 
such  that  \\ys  —  y ||  <  6,  by  Lemma  3.7.1,  there  exists  a  path  v  G  AC  ([0,  5]  :  S )  with 
v  (0)  =  y,  v  (5)  =  ys  with  hiy)  <  c(8).  Let  7  be  the  concatenation  of  7  and  v. 
Then  we  have 

V  (x,  ys )  <  It+s  (7)  =  h  (7)  +  h  iy)  <  V  (x,  y)  +  e/2  +  c  (5) . 

It  suffices  to  choose  5  such  that  c(S)  <  e/2.  The  other  inequality  (and  the  joint 
continuity  with  respect  to  both  variables)  are  proved  in  the  same  way.  □ 


We  can  complete  the  proof  of  the  locally  uniform  LDP  upper  bound  as  follows. 
For  6  >  0,  pick  7  G  AC  ([0, 1]  :  5)  such  that  7  (0)  =  Ho,  7  (t)  G  B  (x,  e),  and  It  (7)  < 
■Jf  (/j,o,x)  +  6.  By  Lemma  3.7.1  there  exists  a  path  v  G  AC  ([0,  e]  :  S)  with  v  (0)  = 
7  (t),  v  (e)  —  x  with  I£[y)  <  c(e),  where  c(e)  — >  0  as  £  — »  0.  Let  7  be  the 
concatenation  of  7  and  n.  We  now  rescale  7  to  obtain  a  new  path:  for  c  =  (t  +  e)  /t, 
define  %  G  AG  ([0,  t]  :  S)  by  %  (s)  =  7  (cs),  s  G  [0,  f].  Then  %  (0)  =  ji0,  %  (t)  =  x. 
Moreover,  by  Proposition  3.5.8,  for  £  sufficiently  small,  It  (%)  <  It+e  (7)  +  8,  and  by 
the  construction  above, 

Jf  (/i0,  a;)  <  h  (7c)  ^  (7)  +  8  =  It  (7)  +  I£  iy)  P  8  <  Jf  (yo,  x)  +  25  +  c  (e) . 

Taking  the  limit  inferior  as  e  — >  0  and  then  sending  <5  — >  0,  (3.7.2)  follows. 

Proof  of  Lemma  3.7.1.  By  Condition  3.1.14.i)  and  Remark  3.1.6,  there  exists  a 
strongly  communicating  path  7  G  AC  iff),  e]  :  S)  such  that  7(0)  =  x,  7(e)  =  y, 
with  constant  speed  U  <  d  ||a;  —  y\\  /e  <  d .  Precisely,  there  exist  F  <  00  and 
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0  =  to  <  t±  <  ■  ■  ■  <  tp  =  1,  such  that 

F 

7  (*)  =  Uv™Mtm- ie,tme)  ( t )  for  a.e.  t  G  [0,e] . 

m= 1 

Since  Ie( 7)  =  (itm£  (7)  -  hm-xe  (7)),  it  suffices  to  bound  each  term  from 

above. 


Recall  from  (3.1.11)  that  for  any  j  G  A fm,  (ej,vm)  <  0.  Let 
b\  =  minm=ij„.jir min^^  \(ej,vm) \  >  0.  Note  that  for  s  G  [tm-i£,tm£),  and  any 
j  G  Afm,  7 j  (tm£)  -  7 j  (s)  =  -  (ej,  vm)  U  (tm£  -  s),  and  thus  7 j  (s)  >  b-JJ  (tm£  -  s). 
Therefore,  by  Definition  3.1.12,  there  exist  constants  c±  >  0,  p±  <  00,  such  that  for 
£  sufficiently  small, 


Km  (7  (s))  >  Cl 


>  ~dUK  (■ tm£  -  s)" 


where  k  =  dp\  <  00  and  ci  =  CibfPl  >  0.  Thus,  by  taking  qVm  =  U,  and  qv  =  0  for 
v  Y  Vm  in  the  first  line  below,  we  have 


L(  7  OMO)) 


< 

< 

< 


,eT1^.5a”(7(s)k(mTw)) 

a”"(7(s)Knttm)+|>(7(s)) 

U]°s(-rr*rtU - v)  ~U  +  C2 

VCiC/^  (tm£  -  s)  J 

-  {k  —  1)  U  log  U  «■'  kU  log  (tm£  -  s)  -  U  (1  +  log  Ci)  +  C2, 


for  some  constant  C2  <  00.  Therefore, 

f*tm£ 

hme  (7)  -  (7)  =  /  L  (7  (s) ,  7  (s))  ds 

<  -C3(U)£log£  +  C4(U)£ 
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for  some  constants  C3  (U) ,  C4  ( U )  such  that  sup^r^i  (C3  ([/)  V  C4  (£/))  <  00.  Sum¬ 
ming  over  m,  we  have  J£(no,y )  <  IE  (7)  <  c(e),  where  c(e)  =  0(e|loge|)  as 

£  — >  0.  □ 

3.7.2  Proof  of  the  Lower  Bound 

For  the  proof  of  the  lower  bound,  take  any  £  >  0  small.  Then  by  the  Markov  property 
for  {/in},  we  have 

Fw>  (/•*"  (t)  =  Xn)  >  P,0  (/•*"  {t-e)eB  (. x ,  e))  •  inf  (/. in  (e)  =  xn) . 

Wn£B(x,e)nSn 

The  LDP  lower  bound  in  Corollary  3.1.11  implies 

lim  inf  -  logPM0  (/ xn  (t  -  e)  e  B  (x,  e))  >  -  J£_£  (/i0,  x) , 

n— kx)  77, 


where 


J£t  (/z0,  x)  =  inf  {It  (7)  :  7  6  d  ([0, 1]  :  S) ,  7  (0)  =  /i0, 7  (t)  e  5  (x,  e)}  . 

The  proof  of  the  lower  bound  will  be  complete  if  we  can  show  both  of  the  following: 

i)  limsupe_0  Jf_e  (no,  x)  <  Jt  (n0,x). 

ii)  The  Local  Communication  Property:  There  exist  a  function  c  :  [0,  00)  — » 

[0,  ex:)  that  satisfies  c(e)  — >  0  as  £  — >  0  and  is  such  that  for  all  £  >  0  sufficiently 
small, 

inf  Pw„  (nn  (e)  =  xn)  >  exp  (-nc  (e)  +  o  (n)) . 

Wn£B(x,e)nSn 
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To  prove  the  first  property,  we  will  use  Proposition  3.5.8.  For  any  5  >  0,  take 
7  £  AC  ([0, 1]  :  S)  such  that  7  (t)  —  x  and  It  (7)  <  J*  (/i0,  x)  +  5.  Take  c  —  t/(t  —  e) 
and  consider  the  path  7C  £  AC  ( [0,  t  —  e]  :  S),  such  that  %  (s)  =  7  (cs),  s  £  [0,  t] . 
Then  7C  (0)  =  po,  7 c  (t  —  s)  =  x.  By  Proposition  3.5.8,  given  5  >  0,  for  £  sufficiently 
small,  It/C  (7 c)  <  A (7)  +  5,  and  we  have 

Jet_e  (p0,  x)  <  It—e  (7c)  =  It/c  (7c)  <  It  (7)  +  &  <  Jt  (ho,  a;)  +  25, 
and  the  conclusion  follows  by  taking  first  £  — >  0  and  then  5—^0. 

Proof  of  local  communication  property.  We  will  use  Condition  3.1.14  and  Lemma 
3.6.1.  Fix  some  wn  £  B(x,e)  D  Sn,  note  that  the  probability  of  pp  (e)  =  xn  is 
no  less  than  the  probability  that  pn  hitting  xn  at  £  by  passing  through  a  given 
discrete  strongly  communicating  path  (j)n  that  connects  wn  and  xn . 

By  Condition  3.1.14,  there  exists  F  <  cx),  0  =  t0  <  t\  <  ■  ■  ■  <  tF  =  1,  {um}^=i, 
and  constants  c\  >  0,  c',  pi  <  00,  such  that  the  image  of  fn  is  Sn  fl  {0  (s)  :  s  £  [0,  e] }, 
for  some  <f  £  AC  ([0,  e]  :  S)  that  satisfies  0  (0)  =  x,  and 

F 

4>  (s)  =  Uvml [tm-ie,tme)  («) ,  a.e.  s  £  [0,  e]  , 

771—1 

with  U  —  d  \\xn  —  wn ||  je  <  d .  Also,  for  s  £  [tm-i£,tm£)  and  large  n,  (0„  (s))  > 
ci(  El  {4>n)j  (s))Pl-  Let  =  4>n  (tm£—).  By  the  Markov  property 

j  £j\f m 

PWn  (h”  (£)  =  xn)  >  n^(m)  (hn  ((Wi  ~  tm)s)  =  ^(m+1)),  and  it  suffices  to  give  a 

771 

lower  bound  for  each  term  in  the  product.  This  will  be  proved  by  comparison  with  an¬ 
other  Markov  process  Zn.  Thus,  without  modifying  the  notation,  we  let  )in  (t)  denote 
the  process  stopped  when  it  first  leaves  the  set  of  points  (0n  (s)  :  s  £  tm£)}. 

For  each  m  and  t  £  [0,  (tm+ 1  —  tm)  e),  define  Zn  to  be  the  jump  Markov  process  with 
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Zn  (0)  =  z(m\  with  the  same  set  V  of  jump  directions,  and  jump  rates 

f  XPi 

nci  El  xj  I  lf  v  =  vrn 
ye  Mm  ) 

nM  if  v  G  V  / vrn 

so  long  as  Zn  stays  in  the  set  {0n  (s)  :  s  G  [tm- i£,  tme)},  and  with  the  process  stopped 
when  it  jumps  off  the  line  segment.  Note  that  A„  (x)  bounds  Ajjm  (x)  from  below  in 
the  set,  while  nM  is  an  upper  bound  on  all  jump  rates. 


Let  p{t)  be  the  probability  distribution  of  Zn  (t):  for  any  x  G  Sn,  px  (t)  = 
P  ( Zn  (■ t )  =  x).  It  satisfies  the  Kolmogorov  forward  equation 


P  it)  ^  ^  (Cjy  &x)  A-xyi 
x,y£Sn 


(3.7.3) 


where 


A 

si-xy 


A n(y-x)  (a?)  if  niy-x)  G  V 
0  else. 


Also,  the  distribution  of  /in  (t)  satisfies 


771  it)  ^  ^  i&y  &x)  A-xy, 
x,y€S„ 


where 


A 

-ti-xy 


A n(y-x)  (x)  if  n  iy  -  x)  G  V 
0  else. 


By  construction 


Pxm  (pn  ((Wl  -  tm)  e)  =  xm+l)  >  izn  ((Wl  -  tm)  e)  =  xm+l)  . 
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If  /  is  the  number  of  points  in  the  discrete  segment  {( f>n  (s)  :  s  G  [tm£,tm+ is)},  then 
by  Definition  3.1.13  l  <  C2ne  for  some  C2  <  00.  The  product  of  the  jump  rates  of 
Zn  along  this  segment  satisfies 

n  (nCl  (  n  xi]  )  ^  ci  ( /!)fcmP1  /n{kmPl~1)l, 

x:x£{<t>n(s):s£  tme,tm+ ie)}  \  VjSA/’m  /  / 

where  krn  =  \Mm\  <  d.  The  lower  bound  in  the  last  inequality  is  achieved  when 
{(f)n  (s)  :  s  G  [tm£,tm+ iff)}  is  a  segmemt  that  ends  up  on  xrn+  \  G  dS,  and  for  all 
j  e  A fm,  Xj  =  1 ,...,/  along  the  segment.  Then  it  follows  from  Lemma  3.6.1  that  for 
£  >  0  sufficiently  small. 

IP Xm  (b  ((bra+1  bn)  £-)  2-m+l  ) 

—  ^P Xm  ((bn+1  bn)  £■)  =  ^m+l) 

>  ^c[  (l\)kmPl  /n{kmPl~1)l  ((bn+1  -  bn)  e)1  exp  (-nM  |V|  (bn+i  -  bn)  e) 

/  7!  \  ^i-1 

>cM— j  exp  (—nM  |V|  e) 

>  exp  (nC2elogc  +  (dpi  —  1)  nC2e  log  (C2e/e)  +  nC2£  log  £  —  nM  \  V\e  +  o  (en)) 
=  exp  (dpin  (C2e  logs  +  0(e))  +  o  (n)) , 

where  for  the  third  inequality  we  used  the  fact  that  the  function  x  1— >  ad  exp  (— nbx ) 
is  decreasing  for  l/n  sufficiently  small,  and  for  the  fourth  inequality  we  used  Stirling’s 
approximation. 

From  this,  we  conclude  Pw  (pn  (e)  =  xn)  >  exp  (—nc  (e)  +  0(e)  +  o  (n))  with 


c(e)  =  O  (e  logs),  as  desired. 


□ 
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Chapter 


Four 


Markov  Chain  Approximation  for 

Quasipotentials 
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This  chapter  studies  Markov  chain  approximations  for  the  quasipotential  function 
associated  with  the  stationary  measure  of  the  mean  field  interacting  particle  systems. 
In  Section  4.1  we  define  the  quasipotential  as  an  optimal  control  problem,  and  state 
basic  assumptions  that  will  be  used  in  the  proofs  in  Sections  4.2.  Section  4.2  contains 
the  proof  of  a  comparison  lemma,  which  allows  us  to  approximate  the  quasipotential 
in  a  small  region  by  solving  a  linear  quadratic  regulator  problem.  In  Section  4.3  we 
present  the  Markov  chain  approximation  algorithm,  following  the  general  framework 
of  [29].  In  Section  4.4  we  construct  different  approximation  schemes,  and  discuss 
the  method  to  solve  the  associated  optimization  problem.  Finally,  some  numerical 
examples  are  shown  in  Section  4.5. 


4.1  Assumptions 


We  focus  on  the  mean  field  interacting  particle  systems  described  in  Chapter  2  with 
K  =  1.  Specifically,  the  empirical  measure  process  {/in  (t)  ,t  >  0}  is  a  cadlag  jump 
Markov  process  that  takes  values  in  Sn ,  with  its  generator  given  by  (2.2.1).  We  will 
denote  Tb  (•)  and  a\3  (•)  (which  are  defined  in  Condition  2.3.3)  simply  as  TtJ  (•)  and 
aij  (■)■ 


Following  Theorem  2.3.2,  {//*}  satisfies  a  functional  law  of  large  numbers  limit 
fi,  that  solves  the  Kolmogorov  forward  equation 


j  fi(t)  =  ^(f)Tr(^(f)), 
I  m(0)  =  Aku 


(4.1.1) 


And  by  Theorem  3.1.10,  {junj  satisfy  the  sample  path  large  deviation  principle  (LDP) 
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with  local  rate  function 


L(x,(3) 


inf 

q:lTq=0 


d  d 

E  E 

1=1  j=l,j^l 


Ctij  (x)  i 


Qij 


Oiij  \X ) 


(4.1.2) 


for  x  G  S  and  0  G  Ad_1={0  G  Rd  :  Yln=\  A  =  0}.  Here  q  is  a  d  x  d  matrix 
with  nonnegative  off-diagonal  entries  and  qu  =  —  i  j&  Qiji  and  1  =  (1, 1)  is  a 
d-dimensional  vector. 


Fix  xq  G  S,  and  let  V  be  the  quasipotential  associated  with  {xo}.  Specifically, 
we  define 


V  (x)  =  inf 


=  inf 


Jo  L  ($  (s) ,  0  (s))  ds:  cpeAC  ([0,  r]  :  S) , 

0  (0)  =  Xq,  0  (r)  =  X,  T  <  OO 

f0T  L  (0  (5) ,  -0  (s))  ds:(peAC  ([0,  t]  :  5) , 
0  (0)  =  x,4>  (r)  =  Xq,  t  <  00 


(4.1.3) 


We  call  L  the  running  cost  of  the  associated  control  problem.  It  is  equivalent  to  take 
the  inhmnm  in  (4.1.3)  over  all  0  G  AC  ([0,  00)  :  S)  such  that  1  unif_»0O  0  (t)  =  Xq,  see 
[10],  Lemma  2. 


By  Lemma  3.1.16,  the  quasipotential  function  thus  defined  is  continuous  (and 
thus  uniformly  bounded)  in  S.  We  also  denote 


V  =  sup  V  (x) . 
xes 


(4.1.4) 


Let  H  be  the  Hamiltonian  which  is  defined  to  be  the  Legendre  transform  of  L : 
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for  x  G  S  and  p  G  Ad  x, 


H(x,p)  =  sup  {{(3,p)~  L(x,(3)}  (4.1.5) 

d  d 

=  E  E  <*ij  (x)  (exp  (pj  -  Pi)-  1) . 

*= 1  7=1  jW 


The  following  result  follows  from  Lemma  3.5.2. 

Proposition  4.1.1.  For  any  x  G  S  fixed,  both  L  (x,  •)  and  H  {x,  ■)  are  strictly  convex 
on  Ad_1. 


For  simplicity  we  will  use  the  following  reduced  coordinates.  The  coordinate 
transformations  Xi  :  (.Ti x(j)  — >  (x±,  ...,Xd-i)  and 

J2  :  (xi,  ...,Xd)  —■ >  (x‘2  —  xi,  ...,Xd  —  x\)  maps  S  and  A^1  respectively,  into 
S  =  {x  G  :  Xi  >  0  and  Yli=i  xi  —  1}  and  Rd_1-  Under  such  coordinate 

transformations,  for  i  =  1, d  —  1,  one  can  denote  Xi  =  Xi,  pi  =  pt+\  —  p\ .  Then 
x,p  G  Md_1,  and  we  can  write  H  as 

d  d 

H{x,p)  =  Y^  (x)  (exP  (Pj- 1  -  Pi- 1)  -  !) ,  (4.1.6) 

i= 1  7=1 

where  po=0.  With  some  abuse  of  notation,  we  still  denote  x,p  as  x,p  respectively, 
in  later  sections. 

Denote  int(<S)  as  the  relative  interior  of  S,  and  note  that  X\  (int(iS))  is  open  in 
Md_1.  For  any  open  set  D  C  Rm,  m  <  d,  and  k  G  N,  denote  Ck  (D  :  M)  as  the  space 
of  k  times  continuously  differentiable  functions  on  D.  We  will  sometimes  abuse 
the  notation,  and  write  /  G  C'fc(int(5)  :  M)  and  g  G  Ck  (Ad_1  :  M)  if  /  o  Zfi1  G 
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C'fc(X(int(5))  :  M)  and  g  o  If1  G  Ck  (Md_1  :  M).  For  /  G  C1  (int(S)  x  Ad_1  :  M), 
we  use  /  and  DPif  to  denote  the  partial  derivative  of  /  o  Zf1  with  respect  to 
Xi  and  the  partial  derivative  of  /  oI2_1  with  respect  to  pt.  For  k  G  N,  higher  order 
derivatives  Dk  and  Dkp  are  defined  in  the  same  way. 

The  equilibrium  point  x$  of  the  law  of  large  numbers  dynamics  (4.1.1)  satisfies 
L  (.x'o,  0)  =  0.  In  other  words, 


OLij  (x0)  -  ajk  (x0)  =  o,  j  =  1, ...,  d. 

*7 k¥=j 

A  direct  computation  gives 

DPjH(x 0,0)  =  ^  Q'ij+i  (x0)  —  X)  aj+i,fc  (x0)  =  0,  j  =  1,  1,  (4.1.7) 

iy^j  +  l  k¥zj  + 1 

Condition  4.1.2.  Assume  that  there  is  a  unique  unstable  equilibrium  point  xq  Eint(S ) 
o/the  law  of  large  numbers  dynamics  (4-1.1),  in  the  sense  that  all  eigenvalues  of  the 
Hessian  DXDPH  (x0,  0)  has  negative  real  parts. 

In  this  chapter  we  focus  on  the  quasipotential  associated  with  the  unique  unstable 
equilibrium  {xo}.  We  also  assume  some  regularity  condition  on  atJ  (•). 

Condition  4.1.3.  Assume  ctij  (•)  G  C3(int(S)  :  M). 

Remark  4.1.4.  Condition  4-1.3  implies  that  H,  and  therefore  its  Legendre  dual  L, 
are  C3  on  int(S )  x  A^1,  in  both  variables.  For  H ,  the  C3— regularity  is  a  direct 
consequence  from  (4-1-6).  For  L,  this  follows  from  the  fact  that  since  L(x,  •)  is 
strictly  convex  on  the  closed  set  Ad_1  (by  Proposition  4-1-1),  the  supremum  in  (4-1-5) 
is  uniquely  attained,  and  thus  L  has  the  same  regularity  as  H  (a  similar  argument 
is  carried  out  in  Lemma  4-2-1  below). 
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Applying  Theorem  2  of  [10],  Condition  4.1.3  implies  that  the  quasipotential  V 
defined  in  (4.1.3)  is  C3  in  some  open  neighborhood  (relative  to  S)  of  Xq  (which  we 
denote  as  D).  We  now  define  the  region  of  strong  regularity  of  V. 

Definition  4.1.5.  For  V  defined  as  in  (4-1-3),  a  relatively  open  set  flo  C  S  is  called 
a  region  of  strong  regularity  of  V  if 

i) .  V  G  C3  (ff0  :  [0,  oo)). 

ii) .  For  any  initial  condition  x  G  Qq,  there  exists  a  unique  the  optimal  trajec¬ 
tory  <f*  in  (4-1-3),  that  satisfies  V  ( x )  =  /0°°  L  (s) ,  — 0*  (s)j  ds,  and  the  image 
{(p*  (s)  :  s  G  [0,  cx))}  C  n0- 

For  general  definitions  and  properties  of  region  of  strong  regularity,  we  refer 
the  reader  to  [19],  Section  6.7.  As  shown  there  in  the  case  of  small  noise  diffusion 
problems,  the  relatively  open  set  flo  is  connected,  contains  some  neighborhood  of  xo, 
and  is  also  dense  in  S. 

The  optimal  control  problem  (4.1.3)  that  defines  the  quasipotential  V  can  be 
associated  with  the  solution  of  a  Hamilton-Jacobi-Bellman  equation.  V  can  be  char¬ 
acterized  (cf.  [10] ,  Section  3)  as  the  maximal  viscosity  subsolution  to  the  PDE 

H(x,DV(x))  =  0,  for  x  G  S\  {^o}  (4.1.8) 

V  (x0)  =  0, 

where  DV=  (DXl  V, ...,  DXd  lV ),  and  H  is  understand  as  in  (4.1.6).  Restricted  to  Qo, 
V  is  a  classical  (C3)  solution  to  (4.1.8)  with  a  local  minimum  at  x$  (Corollary  5  of 
[10]). 
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4.2  A  Comparison  Lemma 


In  this  section  we  provide  a  comparison  lemma  for  two  quasipotentials.  We  will  con¬ 
struct  an  approximation  of  the  quasipotential  (4.1.3),  denoted  as  V (2\  in  a  neigh¬ 
borhood  of  xq,  by  taking  the  approximated  running  cost  associated  with  the  same 
exit  problem.  The  approximate  quasipotential  can  be  solved  explicitly  using  linear 
quadratic  regulators.  Later  in  Section  4.3  we  will  construct  a  discrete  approxima¬ 
tion  of  the  solution  to  (4.1.8),  and  in  particular  its  boundary  value  at  xo  will  be 
approximated  by  in  the  intersection  of  the  lattice  and  a  ball  contains  xq.  In 
what  follows  we  assume  all  the  conditions  stated  in  Section  4.1. 

By  Condition  4.1.3  and  (4.1.6),  we  see  that  H  (x,  0)  =  0,  and 

DkxH  (x,  0)  =  0,  k  =  1,  2  (4.2.1) 

for  any  x  eint(<S).  Thus,  recalling  (4.1.7),  for  (x,p)  near  (x0,  0)  we  have  the  following 
expansion: 

H  (x,p)  =  ^pTAp+^(x  -  x0)T  Bp+^pTBT  (x  -  x0)  + O  (\\x  -  x0f +  \\p\\3) 

=  ^pT Ap  +  (x  -  x0)T  Bp  +  O  (\\x  -  x0\\3  +  ||p||3)  ,  (4.2.2) 

where,  using  (4.1.6), 

Aij  =  DPiDPjH(x  0,0) 

Ylk=l,k^i+1  ak,i+ 1  (®0)  +  Thk=l,k^i+l  ai+l,k  (%o) 

CL+ij+i  (xo)  Qy+i,i+i  (zo) 


if  i  =  j, 

if  i  j , 


(4.2.3) 
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and 


Bij 


Dx,  DPi  H  (x0,  0) 

d  d  d  d 

Y  T-ak,j+i{x0)-  J2  T—aJ+i,k(x0).  (4.2.4) 

k=i,k&+i  UXl  k=i,k&+i  UXl 


Since  H(x,-)  is  strictly  convex  on  Ad  1  by  Proposition  4.1.1,  A  is  a  positive 
definite  matrix.  Define 


(x,  p)  =  -pT  Ap  +  (x  —  x0)T  Bp. 


The  Legendre  transform  of  H ^  takes  the  form 


L^(x,/3)  =  sup 

peAd_1 


{P,p)  ~  (  - pTAp  +  (x  -  xq)1  Bp 

A  A- 1 


=  -  (/3  -  Bt  (x  -  x0))  A  1  {(5  -  Bt  {x  -  x0))  ,  (4.2.5) 


for  x  G  S  and  j3  G  Ad  1 .  We  claim  that  L ^  is  the  true  quadratic  approximation  of 
L ,  in  the  following  sense. 

Lemma  4.2.1.  There  exists  e  >  0,  such  that  for  any  (x,/3)  G  S  x  Ad_1  with 
||x  —  xq||  <  £  and  ||/3||  <  e,  L  (x,  /3)  =  L ^  (x,  j3)  +  O  (||x  —  xo||3  +  ||p||3)  ■ 


Proof.  By  taking  e  smaller  if  necessary,  we  can  assume  without  loss  of  generality  that 
x  Gint(iS).  By  Remark  4.1.4,  L  G  C3(int(5)  x  Ad_1  :  M).  It  suffices  to  show  that  the 
partial  derivatives  of  L  and  at  (xq,  0)  are  equal  up  to  the  second  order.  It  is  clear 
from  the  definition  of  the  equilibrium  point  that  L(xo,0)  =  L ^  (xo,  0)  =  0.  Since 
L  (x,  (3)  =  supp  ((/3,  p)  —  H  (x,  p)),  the  fact  that  H  is  strictly  convex  in  p  (Proposition 
4.1.1)  implies  that  the  supremum  is  attained  at  some  unique  p*  =  p*  (x,/3).  By  the 
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implicit  function  theorem,  the  strict  convexity  of  H  (x,  •)  and  the  fact  that  H  is  C 3 
in  both  variables  (Remark  4.1.4),  p*  is  C 3  in  both  variables,  satisfies  p*  (xo,  0)  =  0 
and 

DPH  (x,p*  (x,/3))  =  /?,  for  (x,/3)  E  int(S )  x  Ad_1. 

Using  the  fact  that 


L  (x,  (5)  =  {0,  p*  (x,  P))  -  H  (x,  p*  (x,  P)) 


and  DH  (xq,  0)  =  0,  we  obtain 


DxL(x  o,0) 
DlxL{x  o,0) 
D2xpL(x  0,0) 
DppL  (x0,  0) 


DPL  (x0,  0)  =  0, 

~DlxH  Oo,  0) , 

-D\VH  (x0,  0)  ( D\VH  (x0,  0)) 
{D2pph(x  o,0)) 


-l 


And  similarly, 


DXLI2>  (i„,0) 
D2„L^(x  o,0) 
Dl0L^(xo,  0) 
£>|j£(2)  (xo>  0) 


DpL(2>  (x0,  0)  =  0, 

-£>L-ff(2)  (*o,0), 

-DlrH^  (x0,0)  (D^hW  (i„,0)) 
(D2„H™  (x0,  0)) 


-1 


.  -1 


The  claim  then  follows  by  noting  that  the  partial  derivatives  of  H  and  H ^  at  (xo,  0) 
are  equal  up  to  the  second  order.  □ 


Note  that  V  (xo)  =  0.  Since  H  (x o,0)  =  0,  H  (x o,  •)  is  nonnegative  and  strictly 
convex,  H  (xq ,p)  =  0  if  and  only  if  p  —  0.  Therefore  within  a  neighborhood  of  Xq,  V 
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is  C3  and  DV  (xq)  =  0  by  (4.1.8).  One  can  then  expand  V  near  xq  as 

V  (x)  —  (x  —  x0)T  P  (x  —  xo)  +  O  (||x  —  x0||3)  .  (4.2.6) 

Since  (xo,  0)  is  an  unstable  equilibrium  for  the  corresponding  Hamiltonian  system, 
P  is  a  symmetric  positive  definite  matrix  (see  Section  4  of  [10]).  We  are  interested 
in  obtaining  the  second  order  approximation  of  V  near  xq,  i.e.  in  identifying  P  in 
(4.2.6).  Define  V ^  as  the  solution  to  the  same  exit  problem  as  in  (4.1.3)  but  with 
the  running  cost  P2'1  instead  of  L: 

vm  (x)  =  inf  |  L<2'  i4,  (S)  •  (s))  *  ■<!>  e  AC  ([0,  r]  :  S)  H  ^ 

|  0  (0)  =  x,  (j)  (r)  =  x0,  r  <  oo  | 

Substituting  the  expression  for  from  (4.2.5),  we  can  rewrite  as 

(  fn  \u  (s)T  A^u  (. s )  ds  :  (j)  =  -BT  (0  -  x0)  —  u,  \ 

H(2)  (x)  =  inf  <  \  .  (4.2.8) 

j  0(0)  =  x,  0(r)  =  x0,r  <  oo  j 

This  is  the  control  problem  associated  with  the  linear  quadratic  regulator,  which  is 
known  to  admit  an  explicit  solution.  Specifically,  it  is  known  (see  e.g.  Chapter  8.2 
of  [37])  to  be  a  quadratic  function  (x)  =  (x  —  xq )T  P  (x  —  xq),  x  G  S,  with  P 
being  the  maximal  solution  of  the  algebraic  Ricatti  equation, 

BP  +  PBt  +  2  PAP  =  0,  (4.2.9) 

where  A  and  B  are  defined  in  (4.2.3)  and  (4.2.4),  respectively. 

In  the  next  theorem,  we  show  that  P  —  P,  and  is  indeed  the  second  order 


approximation  of  V. 
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Theorem  4.2.2.  There  exists  5  >  0,  such  that  for  any  ||x  —  xo||  <  5,  we  have 

\V  (x)  —  V ^  (x)|  =  O  (||x  —  x0||3)  . 

Proof.  Since  DV  (x)  =  2P  (x  —  xo)  +  0(\\x  —  xo||2),  substitute  the  expansion  of 
Hamiltonian  (4.2.2)  into  the  Hamilton- Jacobi  equation  (4.1.8),  we  obtain 

(x  —  x0)T  ( BP  +  PBJ  +  2 PAP)  (x  —  xq )  +  0  (||x  —  x0||3)  =  0, 

and  therefore 

BP  +  PBt  +  2PAP  =  0.  (4.2.10) 

The  maximality  of  P  in  (4.2.9)  therefore  implies  V  (x)  <  V ^  (x)  +  0(\\x  —  a;o||3)- 

To  prove  the  other  inequality,  for  y  in  some  neighborhood  of  x0,  (4.1.8)  and 
the  definition  of  H  via  (4.1.5)  implies  the  optimal  velocity  f3*  =  (5*  (y)  is  uniquely 
attained,  and  satisfies 

{DV{y),P*)-L(y,P*)  =  0.  (4.2.11) 

And  the  optimal  path  ip  of  (4.1.3)  takes  the  form 

V(t)  =  -/T(^(t)),  (4.2.12) 

<p  (0)  =  x. 

By  the  implicit  function  theorem  for  vector  valued  functions,  f3*  (•)  is  at  least  C 2  in 
a  neighborhood  of  rc0,  /3*  (xq)  =  0,  and  near  xq  it  admits  Taylor  expansion  (3*  (■ y )  = 
E  (y  —  xq)  +  0(\\y  —  Xo||2).  Now  recall  the  quadratic  approximation  of  L  near  (xo,  0) 
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in  (4.2.5).  Looking  at  the  0(\\x  —  £o||2)  term  in  (4.2.11),  one  obtains 

EtP  +  PE  ~^{ET  -  B)  A-1  (E  -  Bt )  =  0. 

Using  (4.2.10),  we  see  that  E  =  BT  +  2 AP  solves  the  above  algebraic  equation.  To 
see  the  solution  is  unique,  take  E  =  B1  +  2 AP  +  X  in  the  above  equation,  an  we 
obtain  | XA~xX  =  0,  the  positive  definiteness  of  A  thus  implies  X  =  0.  Therefore, 

/r  (y)  =  ( Bt  +  2 AP)  (y  -  x0)  +  0(\\y  -  x0||2). 

Note  that  U  ( x )  =  (x  —  xq)t  P  (x  —  xq)  is  a  Lyapunov  function  to  (4.2.12).  In 
fact,  there  exists  8,  Co  >  0,  such  that  for  any  \\ip  (t)  —  &o||  <  8, 

=  (^i,t)-x0)T  (-BP-PBT -APAP)((p(t)-x0) 

+  O  (||y>(t)  —  ^o||3) 

=  -2  (( p  ( t )  -  X0)T  PAP  (ip  ( t )  -  Xq )  +  0  (||<£>  (t)  ~  X0||3) 

<  -Co  ||<p(t)  -X0\\2. 

Where  we  use  (4.2.10)  to  obtain  the  second  equality,  and  the  strict  positive  defi¬ 
niteness  of  PAP  to  obtain  the  last  inequality.  Also,  the  strict  postitive  definite¬ 
ness  of  P  implies  there  exists  K\  >  0  and  K2  <  oo  such  that  K\  \\y  —  Xo||2  < 
U  (y)  <  K2  || y  —  xq\\2.  Applying  a  version  of  Lyapunov  exponential  stability  the¬ 
orem  for  quadratic  Lyaponov  functions  (see,  e.g.  Theorem  3.4  of  [32]),  there  ex¬ 
ist  Ci  =  C'i  (4) ,  C2  =  C2  (8)  >  0,  C3  =  C3(8),  such  that  for  ||x  —  xo||  <  8, 
\\p  (t)  —  x0||  <  Ci  ||a;  —  x0||  e_G2*  and  \\<p  (t)||  <  C3  ||a;  —  a;0||  e~C2t .  Taking  this  so¬ 
lution  to  be  the  control  picked  in  (4.2.8),  we  obtain  constants  Cfi,  C3  <  00,  such 
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that 


V ^  ( x )  < 
< 


L(2)  (ip  (s)  ,-<p(s))ds 

[L  (p  ( s ) ,  -0  (s))  +  C4  (||p  (t)  -  ^o||3  +  ||0  (Oil3)]  ds 


<  V(x)  +  C5\\x-x0\\3 
=  V(x)+  Hx-xoll3 


□ 


4.3  Markov  Chain  Approximations 

We  now  adapt  the  methods  in  [29]  and  [4]  to  construct  a  discrete  approxima¬ 
tion  to  the  solution  of  the  optimal  control  problem  (4.1.3).  Following  [4],  we  con¬ 
struct  value  functions  defined  on  lattice  approximations  of  S,  that  solve  discrete 
analogs  of  Hamilton- Jacob i-Bellman  equations.  Before  describing  the  construction 
we  point  out  the  special  features  of  the  model  that  did  not  appear  in  [29]  and 
[4],  First,  the  set  of  all  possible  jump  directions  of  the  Markov  process  (2.2.1)  is 
V=  {ej  —  e*  :  i,j  =  1, ...,  d,  i  ^  j},  which  is  different  from  the  usual  nearest  neighbor 
random  walks  on  square  lattices.  Second,  the  boundary  for  our  exit  problem  is  the 
singleton  {xo}.  We  take  a  sequence  of  sets  Bh  C  Sh  (that  shrink  to  {xo}  as  h  — >  0) 
to  approximate  this  boundary,  and  assign  the  boundary  conditions  on  Bh  by  the 
quadratic  approximation  V^2'1  studied  in  the  previous  section  (which  can  be  obtained 
by  solving  (4.2.9)  for  its  Hessian).  These  differences  will  affect  our  construction  of 
the  Markov  chain  approximations  and  the  convergence  proof. 

We  now  construct  a  discrete  time,  discrete  state  controlled  random  walk  that 
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approximates  the  dynamics  —<fi  =  u  as  h  — >  0. 

For  h  >  0,  we  consider  the  controlled  Markov  chain  £h  with  state  space  Sh=hZdr) 
S.  Since  by  taking  an  affine  map  S  maps  to  some  S  C  Mrf_1,  Sh  can  also  be  identified 
with  hT^1  n  S,  where  Td_1  is  the  d  —  1  dimensional  triangular  (simplicial)  lattice. 
Also,  define  Bh  =  {a:  G  Sh  :  \\x  —  xo||  <  Coh1/3}  ,  where  Co  <  oo  is  a  constant  to 
be  chosen,  and  since  it  will  not  affect  the  convergence  rate  of  the  algorithm,  we  set 
Co  =  1  for  simplicity. 

The  control  space  and  dynamics  can  be  described  as  follows.  In  the  limiting 
calculus  of  variation  problem  (4.1.3),  the  control  takes  value  in  measurable  functions 
on  [0,  cx)).  In  discrete  time  formulation  of  the  Markov  chain  approximation,  for  fixed 
h  >  0,  we  work  with  time  and  state  dependent  controls  {uj  ( x )  :  j  G  N,  x  G  5h}, 
with  u1]  (•)  taking  value  in  the  control  space  Ad_1.  One  can  construct  the  controlled 
random  walk  using  different  class  of  controls.  We  choose  to  work  with  feedback 
controls,  because  it  is  simple,  also  because  the  state  space  Sh  is  finite,  and  the  running 
cost  L  is  strictly  convex  (thus  grows  superlinearly)  in  the  control,  the  optimal  control 
sequence  for  our  discrete  value  function  (introduced  in  (4.3.6)  below)  is  uniquely 
attained. 

Given  the  feedback  control  {uj  (-)}jeN,  we  can  define  a  discrete  time  controlled 
random  walk  £h.  Namely,  for  j  G  N,  £^+1  is  obtained  by  updating  with  transition 
probability  y\Uj  (£j)),  the  time  interpolation  Ath(^,  According  to 

[29] ,  ph  and  A th  should  be  chosen  to  satisfy  the  local  consistency  relations: 

h  vph  (. x ,  x  +  hv\u)  =  uAth  (x,  u ) , 

vev 


(4.3.1) 
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and 

^  || hv  —  uAth  (x,  n)||2pft  (x,  x  +  hv\u)  =  o  (||n||  A th  ( x ,  u))  .  (4.3.2) 

v£V 


Take  a  subset  V*  C  V,  so  that  Ad_1  can  be  decomposed  as  union  of  cones  Ctl 
each  of  which  is  generated  by  d  —  1  linearly  independent  vectors  in  V*,  which  we 
denote  as  Bf 

Ci  =  <  u  G  Ad_1  :  u  =  cvv,  cv>0 
l  v£Bi 

We  also  require  these  cones  are  minimal,  in  the  sense  that  for  any  v  G  v  ^  Cl. 

Given  u  belongs  to  some  Ct,  u  can  be  written  uniquely  as  a  linear  combination 


u  =  uvv  (4.3.3) 

veBi 

for  non-negative  and  we  set  uv  =  0  for  v  G  One  natural  choice  of 

(ph,  A th)  is  to  set 


Ph  (x,y\u) 


l^jj-  if  y  —  x  +  hv ,  v  G  £>* 
0  otherwise 


for  u  G  Ci, 


(4.3.4) 


where  IMIi  =  )C„eV  and 


A th  (u) 


(4.3.5) 


Let  D  ([0,  cx))  :  Md)  denote  the  space  of  cadlag  paths  in  Equipped  with  the 
Skorohod  metric,  this  becomes  a  complete  separable  metric  space.  One  can  also 
take  a  continuous  time  interpolation,  so  that  {^j'jqeN,  {uj  (')lieN  become  elements 
of  D  ([0,  cxd)  :  Md).  They  can  be  constructed  recursively  as  follows.  At  t  —  0,  take 
(0)  =  G  Sh->  uh  (0)  =  uo  (£o)  e  Ad_1,  and  set  A th  (u%)  by  (4.3.5).  For  s  G 
[0,Ath(ug)),  define  (s)  =  £g  and  uh  (s)  =  Uq  (£q  )  •  Suppose  for  some  k  G  N, 
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WW- 1 and  iui}i<k~i have  been  defined) we  set  tk  =  eLo  Afh  iui  (O)- At  tk, 

update  the  value  of  (denote  as  £(?)  using  (4.3.4)  with  the  control  uk_  1  (£fc_i)>  and 
denote  uh  (tk)  =uk  (££).  Let  tk+1  =  E*=o  Ath  {ui  (^));  and  for  s  e  [4,4+ 1),  define 
(s)  —  and  uh  (s)  =  uk  (Zk)-  We  will  switch  between  the  discrete  and  continuous 
time  description  of  £h  and  uh,  whichever  is  more  convenient. 

Given  a  feedback  control  {uj  (-)}jeN  and  a  controlled  random  walk  {'4}JeN,  we 
can  decompose  the  random  walk  into  a  drift  part  and  a  local  martingale  with  mean 
zero:  =  E  (f£|u£_i  (fk-i))  +  ™k,  where  E  ($ K-i  ($- 1))  =  huk- 1/  |K-i||r 

Now  consider  the  following  optimal  control  problem  on  <S4  For  a:  G  Bh ,  we  set 
(x)  =  ( x ).  For  x  E  Sh\Bh,  the  value  function  is  defined  by 

rrh 

Vh(x)=  inf  Ex  / 

[0,oo):  JO 

where  Ex  denotes  the  expectation  conditioned  on  £h  (0)  =  x,  and  the  exit  time 
Th  =  inf  {s  :  (s)  E  Bh }  . 

It  is  shown  in  [4],  Section  3  (see  also  Chapter  5.8  of  [29]),  that  Vh  solves  the 
discrete  dynamic  programming  equation 

Vh  (x)  =  inf  ph  (x,  x  +  hv\u)  Vh  (x  +  hv)  +  L  (x,  — u )  A th  (u)  ,  x  E  Sh\Bh, 

«g  Ad— 1  ■ ' 

_vev 

Vh  (x)  =  V{2)  (x) ,  x  E  Bh ,  (4.3.7) 

where  ph  and  A th  are  given  by  (4.3.4)  and  (4.3.5),  respectively.  This  is  our  starting 
point  for  numerical  approximations.  Note  that  the  discrete  dynamic  programming 
equation  also  holds  when  x  is  on  dS. 


L(Ch  (s) ,  -uh(s))ds  +  V^2\e{rh))  ,  (4.3.6) 
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Vh  can  also  be  interpreted  as  a  solution  to  a  discrete  HJB  equation.  For  u  G  Ad_1, 
let  {«4„ev  be  given  by  (4.3.3),  and  we  define  to  be  the  weighted  finite  difference 
operator,  given  by 


DvJ  (x)  =  XX 

vev 


f  {x  +  hv)  -  f  ( x ) 
h 


x  G  S' 


for  every  bounded  function  /  :  Sh  — >  M.  Then,  subtracting  Vh  (x)  from  both  sides 
in  (4.3.7),  and  dividing  by  A th  (u),  one  obtains 


0  =  inf  \Dyh(x)  + L(x:-u)}  ,  x  eSh\Bh,  (4.3.8) 

u£Ad~ 1  L  J 

vh  (x)  =  V(2)  (x),  xe  Bh. 


4.4  Numerical  Approximations 


We  now  construct  numerical  schemes  for  the  Markov  chain  approximation  that  satisfy 
local  consistency  properties  (4.3.1)  and  (4.3.2).  The  controlled  random  walk  should 
be  defined  in  such  a  way  that  the  transition  probability  has  a  simple  expression,  the 
dynamic  programming  equation  (4.3.7)  can  be  explicitly  solved,  and  the  data  can 
propagate  quickly  from  the  boundary.  We  discuss  below  two  choices  of  controlled 
random  walks  that  satisfy  these  criteria. 

Note  that  in  the  problem  (4.3.7)  there  are  two  layers  of  inhmization:  the  first 
inhmization  is  over  all  the  controlled  jump  rates  (in  the  definition  of  L  (x,  — u )  in 
(4.1.2))  that  lead  to  a  given  drift  vector  u,  the  second  inhmization  is  over  u  G  Ad_1. 
We  start  with  a  lemma  that  simplifies  the  problem  into  a  single  inhmization  over  the 
controlled  rates. 
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Lemma  4.4.1.  For  any  h  >  0  and  x  G  Sh 


inf 

i£Ad~ 1 


( x ,  x  +  /w|w)  (a:  +  hv)  +  L  (x,  —  u)  Ath  ( u ) 

uev 

inf  <  ph  \  x.x  +  hv  \  —  V'*'  vav  )  Vh  (x  +  hv) 

^ v|  life  V  tTv  ) 

+(S“”(xK(^))Atl(_S^)}' 


(4.4.1) 


{clu}E[0, 


(4.4.2) 


Proof.  Recall  that  L  (x,  - u )  =  inf{Su}:_u=E,jSii  £)weV  av  (x)  £  For  any  fixed 


u 


G  Ad  1,  since  the  function  {a„}  i— >• 


v&V  av  VXJ  ^  V  av(x) 


(x)  £  (  ay  .  )  is  strictly  convex  on 


[0,oo)lvl,  the  inhmum  in  the  definition  of  L(x,—u)  is  uniquely  attained  at  some 
{a*  (x,u)}  G  [0,  oo) such  that  —  u  =  y^eVua*.  Therefore, 

yy  ph  ( x ,  x  +  hv\u)  Vh  ( x  +  hv)  +  L  (x,  — u )  A th  (u) 
vev 

—  Y  ph  (x,  x  +  hv\u)  Vh  (x  +  hv)  +  (  V  av  (x)  £  ( — )  |  Ath  (u) . 

^  '  OLv  (x)  >  1 


vev 


\vev 


Taking  the  inhmum  over  u  G  Ad  1  one  obtains  the  left  hand  side  of  (4.4.1)  is  greater 
than  or  equal  to  the  right  hand  side. 

To  prove  the  other  inequality,  take  any  {a„}  G  [0,  oo) ,  and  let  u  G  Ad_1  be 
given  by  —  u  =  vav.  Then  we  have 

y~~]  ph  (x,  x  +  hv\u)  Vh  (x  +  hv)  +  L  (x,  —u)  A th  (u) 

v£V 

<  Y ph  (. x ,  x  +  hv\u)  Vh  (x  +  hv)  +  (  V  av  (x)  £  ( — )  A th  (u) . 

V  \av{x)J 


Inhmize  over  all  {a„}  G  [0,  oo)lvl,  one  obtains  the  reverse  inequality. 


□ 


132 


118 


We  next  state  a  result  which  gives  the  structure  of  the  minimum  of  the  opti¬ 
mization  problem  (4.4.1).  The  strictly  convexity  and  uniform  superlinear  growth  of 
L(x,-)  implies  the  local  minimum  of  (4.4.1)  is  uniquely  attained,  and  is  also  the 
global  minimum.  We  start  with  a  result  in  convex  analysis  proved  in  [4]. 

Lemma  4.4.2  (Lemma  7.1  of  [4]).  Let  g  :  [0,  oo)  — >  M  be  strictly  convex  and  satisfy 
g  (0)  >0  and  lim^oo  g  (z)  /z  —  oo.  Then  there  is  a  unique  point  x  G  (0,  oo)  that 
achieves  the  infimum  in  inf z>o  g  (z)  /z.  x  is  also  the  unique  local  minimum  of  the 
map  z  i— >•  g  (z)  jz. 

Lemma  4.4.3.  There  exists  a  unique  local  minimizer  of  the  optimization  problem 
(f.f.l),  which  is  also  the  global  minimizer. 

Proof.  For  z  >  0,  define 

g(z)  =  inf  z  ph  (x,  x  +  hv\u)  Vh  (x  +  hv)  +  L  (x,  —  u)  Ath  (u) 

u£Ad-1:\\u\\,=z  Z ' 

1  Lv6V 

=  inf  uvVh  (x  +  hv)  +  L  (x,  —u)  h  , 

weAd-l:|H|,=z  ^ 

"  111  Lt>ev  J 

where  we  used  the  definition  4.3.4  and  4.3.5  to  obtain  the  last  equality.  By  Proposi¬ 
tion  4.1.1,  we  see  that  g  is  the  minimum  of  a  strictly  convex  function  subject  to  an 
affine  constraint,  which  is  itself  strictly  convex.  Moreover,  for  fixed  z,  there  exists 
a  unique  u*  E  Ad_1,  =  z  such  that  the  minimum  is  attained.  For  x  f  Bh, 

g  (0)  =  L  (x,  0)  h  >  0.  The  superlinear  growth  of  L  (see  Proposition  3.5.4)  im¬ 
plies  that  linifc_1.0O  <7  (z)  /z  —  00.  The  conclusion  then  follows  by  applying  Lemma 


4.4.2. 


□ 
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4.4.1  Iterative  solver:  The  widest  neighborhood  structure 

We  employ  Gauss-Seiclel  iterative  method  to  solve  the  dynamical  programming  equa¬ 
tion  (4.3.7)  (see  Section  4.5  for  more  details).  By  Lemma  4.4.3,  if  one  can  find  a 
local  minimize!'  to  (4.3.7),  it  gives  the  global  minimum.  The  strategy  to  solve  the 
minimizer  in  (4.3.7)  or  (4.4.2)  is  to  solve  the  inhmization  in  each  cone  C*  (introduced 
in  Section  4.3),  and  then  minimize  over  all  cones.  To  put  it  in  more  detail,  the 
first  step  is  to  consider  the  relative  interior  of  a  cone,  and  look  for  local  minimum 
as  if  there  were  no  constraints.  We  will  do  this  computation  in  Section  4.4.1  and 
4.4.2.  If  the  inhmum  of  this  problem  is  finite,  some  nonnegativity  constraints  are 
tested  for  each  candidate  minimizer.  If  these  constraints  are  satisfied,  then  a  local 
minimum  is  found.  If  none  of  the  candidate  minimizer  satisfies  the  constraints,  then 
the  unique  minimizer  must  be  on  the  boundary  of  one  of  the  cones.  We  then  search 
these  boundaries  that  form  lower  dimensional  (d  —  2,  d  —  3,...)  hyperplanes  of  W1. 
The  procedure  ends  by  searching  through  all  the  lower  dimensional  boundaries  until 
the  local  minimum  is  found. 

We  now  focus  on  the  optimization  problem  (4.4.2)  inside  a  given  cone.  Different 
choices  of  V*  lead  to  different  {Ci}  (which  we  call  ’’neighborhood  structures”),  and  the 
choice  should  balance  the  tradeoff  between  the  explicit  solvability  of  the  inhmizer  in 
(4.4.2),  and  the  number  of  constrained  minimization  problems  that  must  be  solved  as 
dimension  grows.  We  present  below  two  different  neighborhood  structures  in  Section 
4.4.1  and  4.4.2. 

We  start  with  a  neighborhood  structure  with  the  widest  possible  cone,  so  that 
boundary  data  can  propagate  quickly  through  the  iteration.  The  disadvantage  of 
this  approach,  as  we  will  see  below,  is  that  the  inhmization  problem  becomes  more 
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complicated  as  d  gets  large. 


Take  a  set  of  vectors  (wi, ... ,Wd )  =  (e2  —  ei,  63  —  e2, ei  —  e^),  so  that  any  d  —  1 
of  them  form  a  basis  for  Ad_1.  For  any  x  G  Sh\Bh,  take  V*=  {wi  :  i  —  1,  ...,d}. 
Since  V*  C  V,  for  each  v  G  V*,  we  use  dt]  to  denote  av  in  (4.4.2)  if  v  =  ej  —  et.  Given 
u  G  Ad_1,  we  can  write 

d  d 

=  (ej  -  (;i) 

i=l  j=\,j^i 
d  j- 1  / 1-1 

=  ^  X! ( X! ,r/'- 

J=1  i= 1  \  k=i 

d—  1  /  k  d  d  k  \ 

-  E  I]  “o ) 

/c=l  \  2=1  J=/c+l  i=/c+l  J  =  1  / 


Suppose  u  belongs  to  the  positive  cone  spanned  by  B\=  {wi, Wd-i},  the  other 
cases  can  be  treated  similarly.  Then  although  the  choice  of  {dtj}  in  the  equation 
above  is  not  unique,  Yli=i  Y^j=k+ 1  —  Yli=k+ 1  Ylj=i  ®ij  is  uniquely  determined  for 
k  =  1, ...,  d  —  1.  Recalling  uv,  we  see  that 


d- 1 


l«lli  = 


au  -  S  5Z 


j)a 


ij- 


(4.4.3) 


k=  1 


,  i=l  j=fc+l 


*=fc+l  j=l 


i=l  1=1;1/* 


We  construct  the  controlled  Markov  chain  such  that  for  k 


ph  (x,  x  +  huifclu)  = 


Ed  sr^k  —  sr^k  \—yd 

i=k-\- 1  A—/ 7=1  2-^/j=k 


U 


=  1, ...,  d  —  1,  take 
if  x  +  h.Wk  G  Sh 


and  ph  ( x,y\u )  =  0  otherwise.  Also,  set 


A th  («) 


1 
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One  can  check  such  a  choice  of  ph  and  A th  satisfies  (4.3.1)  and  (4.3.2).  We  then 
solve  the  following  unconstrained  minimization  problem 


d—  1  sr^d  sr^k  -  sr^k  sr^d 

i  2^i=k+ 1  2^7=1  ai?  —  2^7=i  2^7= 


j=k+ 1  ai? 


{aij  } 


fc=l 
d  d 

ay  (-0 

i=l 


U 


V  (x  +  hwk) 


h 


aij  (X)  J  J  IMIl  j 


(4.4.4) 


where  is  dehned  by  (4.4.3),  and  then  check  if  the  optimal  control  u  of  each 
candidate  inhmizer  stays  in  the  interior  of  the  positive  cone.  If  it  does,  then  the 
unique  local  minimum  of  the  constaint  optimization  problem  is  found,  and  we  obtain 
the  global  minimum  of  (4.4.2). 


Let  A  =  HwIIj,  we  introduce  the  Lagrange  multiplier  A  G  M.  and  study 


£ 


Ed  sr-^k  —  sr^k  sr~^d 

i=k+ 1  A—/ 7=1  Q'ij  2-^i= 1  2^j=, 


j=k+ 1  a*i 


.  k= 1 
d  d 

£  £ 

,7=1 


A 


V  (x  +  hwk)  + 


[x) 


h  A h  ( /•  _  A 

aij(x)J)A+ 


Recall  that  for  a;  >  0,  £  (x)  =  a;  log  a;  —  x  +  1.  At  each  local  minimum  of  (4.4.5),  the 
derivatives  with  respect  to  {dij}  are  zero,  which  leads  to: 


j- 1 

-  ^2  V  (x  +  hwk)  +  h  log 

k=i 


aij 

(*^) 


i—1 

^  V  (x  +  hwk)  +  h  log 

k=j 


aij 

c^ij  (*r) 


(j  -  i)  Ah 


(j  -  i)  Ah 


0,  for  1  <  i  <  j  <  d 


0,  for  1  <  j  <  i  <  d 


Thus  we  can  parameterize  for  some  A  G  M, 


dij  =  a^  ( x )  e^x+^=iv{-x+hWkVh,  for  1  <  %  <  j  <  d 
dij  =  a^  {x)  eb-d^-£r4  v(x+hWk)/h,  for  i  <  j  <  i  <  d. 


(4.4.6) 
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Denote  C  =  Yli=i  aij  (+)•  Substituting  (4.4.6)  into  (4.4.4),  we  have 


1 

A 


EE  a.^  e0'-*)A+E i=<  V(x+hwk)/h  (E  V  (x  +  /m+) 

L  1=1  i=l 
d  1-1 

(i-*)A+Efc=i  V(x+hwk)/h  ' 


,  /c=i 


+/l  EE  otij  (x)  £ 

j= 1  i=l 
d  d 

+ 55  55  0) e 

j=l  i=j+l 
d  d 

+h  EE  Q!ij  (+)  i7 1  e 


h 

A 


1=1  i— 1+1 
d  1-1 


(1-bA-EU-  V(x+tofc)/h  (  1/  (x  +  hwk) 

k=j 

,(l-*)^-Efc=<  V(x+hwk)/h 


EE  (j  -  i)  X GLij  (x)  e(i-i)x+'E*=i  V(x+hwk)/h 

Ll=i  »=i 

EE  aij  (x)  g(l-*)^+Efc=i  V(a:+/iM)fc)//i 

1=1  1=1 

^  Cz  “  *)  A«o  (+)  e(j-*)A-E^  y(x+tofc)//l 

l=i *=i+i 

J5  aij(a;)e(i"i)A“E^y(3:+^)/,l  +  C' 

l=i *=l+i 


For  b  —  1, ...,  d  —  1,  define 


d—b  d 

Kb  =  'y  ^  ^  "  5j-i=bCXij  (x)  eEfc=i 
*=i  l=i 
d—b  d 

Ii-b  =  ^  ^  ,  5j—i=-bOtjj  (x)  e  E 
l=i  *=i 


.  V(x+hwk)/h 


V  (x+Zitu*. ) /^ 


We  can  rewrite 


d —  1  /  k  d  d  k  \  d —  1  d— 1 

A  =  -  55  ( 55  55  %■  -  51  55  )  =55  6A-^'bA  -  55  6A^fcA 

fc=l  \  2=1  j=/c+l  i=fc+l  j=l  /  6=1  6=1 
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Then  (4.4.4)  can  be  further  reduced  to 


h 

A 


d- 1 

X>- 

.  6=1 


d— 1 


-b 


-b\e~bX  -  e-bA 


)  +  5>6(&a 


ebA  -  ebA 


)+c 


E&=i  ATbe-bA  +  ES  K^X  ~  C 
Et;  ^-be-ftA  -  Et;  6^ 


6=1 
d_1  rv.  „b\ 


=  h  -A  - 


Minimizing  this  expression  with  respect  to  A,  one  looks  for  A  that  satisfies 


-  Et i  b2K-be~bX  -  Eti  ^ebA  E£  ATbe-bA  +  Eti  KbebX  -  C 


=  0. 


Eti1  ^-^-ftA  -  E£  bKbe 


d- 1 


36A 


Take  f3  =  eA,  we  then  look  for  the  positive  solutions  [3  to  either 


b2Kb(3b  —  0 

6=— (d— 1) 


^  A7,/3b  -  C  =  0.  (4.4.7) 

6=— (d— 1) 

Recall  {Kb}  are  nonnegative,  so  the  first  equation  has  no  positive  root.  We  then 
solve  for  the  2  (d  —  1)  roots  of  (4.4.7).  Since  the  single  variable  polynomial  equations 
(4.4.7)  has  only  two  sign  changes  between  consecutive  real  coefficients,  it  follows 
from  Descartes’  rule  of  signs  that  it  has  at  most  two  positive  real  roots.  For  each 
positive  real  root  /?,  one  computes  A  =  log/3,  and  then  obtains  the  value  of  { }  by 
(4.4.6).  If  the  constraints 


d  k  k  d 

^  ^  ^  ^  Q>ij  ~  ^  ^  ^  ^  Q*ij  —  /c  =  1, d  —  1, 

i=k+ 1  j= 1  i= 1  j=k-\- 1 


(4.4.8) 
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are  satisfied,  one  computes  the  cost 


h 


-log/3  - 


EtJ  bK.be~^  -  EtJ  bKbe *  J 


~h  log/3, 


for  the  local  minimizer. 


4.4.2  Iterative  solver:  The  narrowest  neighborhood  struc¬ 
ture 

Here  we  present  another  possible  neighborhood  structure  for  the  iterations.  Recall 
that  in  the  case  of  the  widest  neighborhood  structure,  for  each  cone  we  need  to  solve 
an  algebraic  equation  (4.4.7)  with  degree  2d  — 2,  which  becomes  more  complicated  as 
dimension  grows.  For  the  following  narrowest  possible  neighborhood  structure  the 
corresponding  equation  is  always  quadratic,  but  at  each  x  e  Sh\Bh  we  need  to  solve 
constrained  inhmization  problems  for  a  large  number  (grows  as  O  ( d 2))  of  subregions 
(cones),  as  d  gets  large. 

Let  {ej}^=2  be  defined  by  e*  =  e*  —  e±,  and  we  set  e\  =  0.  {<=2 , ...,  ed}  forms  a  basis 
of  Ad_1.  For  any  x  e  Sh\Bh  take  V*  =  V.  Using  the  same  notation  as  in  Section 
4.4.1,  for  u  €  Arf_1  one  can  write 

d  d  d  d 

—u  =  'y  ^  "  dij  (e.j  —  ei)  =  'y  ^  y  "  dij  (ej  —  A) 

i=l  j=l,j^i  i=l  j=l,j^i 

d  /  d  d  \ 

=  dij  |  e,. 

*=2  \j=l,j^i  j=l, / 

Suppose  without  loss  of  generality  that  u  belongs  to  the  positive  cone  spanned  by 
Bi  =  {e2,...,ed}.  Then  | M|j  =  -  Ei2  (E?=ij^  ~  E?=i %)  =  Ei2  “ 
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solve  the  following  unconstrained  minimization  problem 


inf 

I 


Ed 

i=ij 


,j¥=i  ah' 


Ed 


ai* 


V(x  +  he-i) 


(4.4.9) 


and  then  check  if  the  optimal  control  of  each  candidate  inhmizer  stays  in  the  interior 
of  the  positive  cone.  If  it  does,  then  the  unique  local  minimum  of  the  constained 
optimization  problem  is  found,  and  we  obtain  the  global  minimum  of  (4.4.2). 


Letting  B  =  Hnl^,  we  introduce  the  Lagrange  multiplier  A  G  M.  and  study 


d 


£ 


Ed  -  vd 

a li  2-^j= i,. 


,j¥=i  aii 


1=2 

d  d 


B 


J=i 


(x) 


F  (x  +  hei )  + 
h  A  h 


B  +  B 


^  ^  °ii  b 

3=  2 


i=2 


(4.4.10) 


At  each  local  minimum  of  (4.4.9),  the  derivatives  with  respect  to  {(%}  are  zero, 
which  leads  to: 


14  (x  +  he.j) 

)  +  V{x 

+  hei)  +  h  log- 

&ij  \p^) 

-V( 

x  +  hej 

)  +  h  log  J 

r-Xk 

(x 

) 

V 

(x  +  hei 

)  +  h  log 

-  +  A  h 

«il  (x 

) 

0,  for  2  <i,j  <d,i^j 
0,  for  j  =  2, ...,  d 
0,  for  i  —  2, ...,  d. 


Thus  we  can  parametrize  for  some  A  6  R, 


Oij  =  Oij  (■ x )  e^+he^-Vix+he^/h^  for  2<i,j<d,i^j 

d\ j  =  aij  (x)  ex+v ( x+hej)/h ?  for  j  =  2, ...,  d 
dn  =  aii  (x)  e-x-v(x+hei)/h ,  for  —  2, ...,  d. 


(4.4.11) 
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Substituting  (4.4.11)  into  (4.4.9),  we  obtain 


1 

B 


d 

[-ay  (x)  ex+v{x+h^)/hV  {; x  +  he-j)  +  honj  (x)  i  (e*+n*+^)/ft)  ] 

-1=2 


d 

+  (a)  e~x-v{x+Mi)/hV  {: x  +  hii)  +  han  (x)  i  (e-^-^(*+^)/ft)] 

i=2 

d  d 

+E  E  [Oij  (x)  eiVix+he^-Vix+M^/h  +  +  y  (x  +  ^)) 

j=2  j=2,j^i 

+haij  (x)  £  . 


Note  that 


hctij  (x)  i  (e(y(s+^)-y(*+^))M) 

=  ay  (x)  e(^(*+^)-^(*+h«i))/ft  (~v(x  +  het)  +  V  (x  +  he,)) 

-hctij  (x)  e(v(x+hij)-v(x+hi*»/h  +  hctij  (x)  • 

Thus  if  we  define 

d 

K\  =  J2aij(x)ev{x+h*j)/h,  (4.4.12) 

1=2 
d 

K2  =  J2ai^x)e~V{X+Ml)/h  (4.4.13) 

i= 2 

c  =  51  Z)  av(x)-J2  Y1  a.j  ^  e (V(x+hej)-V(x+hei))/h, 

i= 1  1=1,1#  *=2  1=2, 1# 

we  can  write 

d  /  d  d  \ 

B  —  —  E  (  E  E  “«] 

*=2  \1=1,1#  1=1;!/*  / 
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And  one  can  further  reduce  (4.4.9)  to 

^  {-e~xK2X  +  exK1X  -  e~xK2  -  exK1  +  C) 

,  (  ^  e~xK2  +  exK1-C\ 

V  e~xK2  -  exK1  j  ' 

Minimizing  this  expression  with  respect  to  A,  one  looks  for  A  that  solves 

(e~xK2  +  exK1  -  C )  (e~xK2  +  exK1)  _ 

(e~xK2  -  exKi)2 

Since  e~xK2  +  exKi  >  0,  one  obtains 

e~xK2  +  exK1  -  C  =  0, 


as  the  local  minimum. 
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4.4.3  The  lower  dimensional  hyperplanes 

To  summarize,  in  the  previous  two  subsections  we  solve  the  infimization  of  (4.4.2) 
in  the  relative  interior  of  each  cone.  The  method  is  to  use  Lagrange  multipliers  to 
first  solve  the  corresponding  unconstrained  problem,  and  then  test  the  nonnegativity 
constraints  ((4.4.8)  or  (4.4.15),  depending  on  the  neighborhood  structure)  for  each 
candidate  minimizer.  If  none  of  the  candidate  minimizer  satisfies  the  constraints, 
we  will  then  need  to  search  lower  dimensional  boundaries  of  the  cones,  until  a  local 
minimum  is  found. 

On  each  lower  dimensional  boundary,  we  need  to  solve  a  constrained  optimization 
problem,  with  additional  affine  constraint  added  to  (4.4.2).  If  we  still  use  Lagrange 
multiplier,  the  calculation  of  the  local  minimum  is  very  complicated,  especially  when 
the  dimension  gets  further  lower  (which  adds  more  constraints).  In  this  subsection 
we  discuss  solving  the  constrained  optimizations  on  these  lower  dimensional  hyper¬ 
planes,  using  the  results  in  Section  4.4.1  and  4.4.2. 

We  show  that,  for  k  —  1,  ...,d—  2,  to  determine  candidate  minimizers  of  (4.4.2) 
on  some  k  dimensional  boundaries,  it  suffices  to  use  jointly  the  candidate  minimiz¬ 
ers  of  (4.4.2)  in  the  relative  interior  of  a  cone,  and  the  solution  of  certain  algebraic 
equations.  The  advantage  of  this  approach,  is  that  solving  the  roots  of  algebraic 
equations  can  be  implemented  much  more  efficiently  than  solving  constrained  opti¬ 
mization  problems,  especially  in  high  dimensions. 

To  illustrate,  let  us  assume  for  now  that  we  use  the  narrowest  neighborhood 
structure  for  the  interation  described  in  Section  4.4.2  (the  other  case  can  be  treated 
in  the  same  way).  Suppose  we  need  to  solve  the  constrained  minimum  on  a  d  —  2 
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dimensional  boundary  of  the  positive  cone  generated  by  {e2, id}.  For  i  =  2 , 
denote  rq  =  Y2j=i  j+i  ®ij  —  Xq=i  hji,  and  suppose  the  boundary  is  given  by 
{u  G  A^1  :  «2  >  0,  ...,Ud- i  >  0,Wd  =  0}.  We  need  to  solve 


inf 

uGAd_1:u2r..,«d_i>0 

«d=0 


inf 

«d=0 


d 

(x,  x  +  hej|u)  (x  +  he}  +  L  (x, 


i=2 

r! 


— u )  Ath  (u) 

(4.4.17) 


If  we  solve  the  corresponding  unconstrained  infimization  problem,  the  candidate 
minimizers  are  given  by  (4.4.16). 


We  now  claim  the  following  result.  By  changing  the  value  of  Vh  (. x  +  hid),  to  a 
particular  value  V*  (if  it  exists),  then  (4.4.17)  equals  to 

inf 

u£Ad—1:u2,...,Ud—i>0 

Note  that  (4.4.18)  can  be  solved  using  the  results  in  Section  4.4.2.  First  obtain  the 
candidate  minimizers  of  the  unconstrained  problem  by  (4.4.14),  and  then  check  the 
nonnegative  constraints  U2,...,Ud-i  >  0.  The  value  of  V*  is  determined  by  jointly 
solving  the  algebraic  equations 

\  C  ±  VC2  -  4KJA 

e  =  - . 

2/7, 

d—  1  d—  1 

yi  hdj — y]  hjd  =  o, 

3= 1  3=1 

where  {a,jd  (A,  17*)}  and  {a3d  (A,  17*)}  are  specified  by  (4.4.11),  K\  (V*)  and  K2  (17*) 
are  given  in  (4.4.12)  and  (4.4.13). 


d- 1 

E 

i= 2 


Uj 


Ud 


U 


-V  (. x  +  hii)  +  T1— 77-V*  +  L  (x,  -w) 


h 


u 


U 


(4.4.18) 


To  see  the  claim  is  true,  note  that  for  each  such  solution  17*,  any  candidate 
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optimal  control  u*  to  (4.4.18),  obtained  via  the  computation  in  Section  4.4.2,  satisfies 
u*d  —  0  (see  (4.4.15)).  Therefore  the  value  of  (4.4.18)  is  less  than  or  equal  to  (4.4.17). 
The  other  inequality  is  straightforward. 

This  argument  works  for  lower  dimensional  (d  —  3,  d  —  4, ...)  boundaries  as  well: 
one  changes  a  subset  of  \Vh  (x  +  he*)}  to  specific  values,  that  are  determined  by 
solving  a  larger  systems  of  algebraic  equations  jointly,  and  use  the  solution  to  the 
unconstrained  optimization  problem  in  a  cone  in  Section  4.4.1  and  4.4.2.  Therefore 
when  search  the  local  minimizer  on  a  lower  dimensional  boundary,  we  use  the  above 
recipe  and  convert  the  problem  into  solving  a  system  of  algebraic  equations. 


4.5  Numerical  Results 


In  this  section,  we  present  numerical  results  obtained  using  the  Markov  chain  ap¬ 
proximation  algorithm  described  in  Section  4.3. 


In  the  actual  implementation  of  the  algorithm,  iterate  the  dynamical  program¬ 
ming  equation  by 


inf 


ueAd~1 


( x ,  x  +  hv\u)  Vj1  ( x  +  hv)  +  L  (x,  — u )  A th  (u) 

_vev 


(4.5.1) 


Therefore  Vj1  (x)  can  be  interpreted  as  the  minimal  cost  of  the  j— step  optimal  control 
problem  with  terminal  cost  Vq.  If  we  take  the  initial  data  V0h,  such  that  for  x  e  Bh , 
Vq  (x)  =  V ^  (x),  and  V0h  (x)  =  oo  for  x  G  Sh\Bh ,  then  the  optimal  control  is 
forced  to  move  toward  the  boundary  set  Bh  quickly,  and  thus  the  boundary  data  can 
be  learned  and  propagated  into  the  domain  quickly.  Moreover,  the  optimal  control 
interpretation  implies  that  Vj1  (x)  is  monotone  decreasing  in  j,  and  converges  to  the 
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maximal  solution  of  dynamical  programming  equation  (4.3.7). 

As  is  described  in  Section  4.2,  the  quadratic  approximation  V ^  can  be  obtained 
by  solving  the  algebraic  Riccarti  equation  (4.2.9)  for  its  Hessian.  We  use  the  Matlab 
build  in  function  to  obtain  the  maximal  solution  of  (4.2.9). 

We  choose  the  initial  condition  to  be 

[  V™  (x)  x  G  Bh, 

V0h(x)={ 

j  M  x  £  Sh\Bh , 

where  M  G  (V,  oo)  is  a  very  large  number,  so  that  the  iteration  (4.5.1)  converges 
quickly. 

For  simplicity,  we  restrict  our  numerical  example  below  to  the  case  d  —  3,  and 
embed  S  C  M3  into  G=  {(aq,aq)  :  aq  >  0,  aq  >  0,  aq  +  X2  <  1}  by  taking  an  affine 
map.  We  also  focus  on  the  nearest  neighborhood  structure  described  in  Section 
4.4.2. 


We  use  Gauss-Seidel  iterative  method  for  our  numerical  experiments.  The  Gauss- 
Seidel  iteration  goes  as  follows.  During  the  first  iteration,  we  enforce  the  value 
function  to  be  less  than  or  equal  to  M  by  letting 


inf 


«g  Ad_1 


(x,  x  +  hv\u)  Vq  (x  +  hv)  +  L  (x,  — u )  A th  (u) 

v£V 


AM. 


Suppose  the  value  {  V- 1  (x)}  was  assigned  to  each  x  G  Sh,  at  the  (j  +  l)th  iteration,  we 
proceed  by  sweeping  through  the  grid  along  six  possible  jump  directions  in  counter¬ 
clockwise  order  (such  as  sweeping  successively  along  the  direction  e 2  —  ei,  —  e\, 
63  —  e2,  ei  —  e2,  ei  —  e^,  e 2  —  63).  During  each  sweep,  we  update  sequentially  the  value 
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at  x  e  Sh\Bh,  by  solving  (4.4.2),  and  the  new  value  will  be  substituted  immediately 
for  calculating  (4.4.2)  at  the  neighboring  points. 


Example  4.5.1.  The  first  example  we  study  is  a  non-interacting  n— particle  system 
with  3  states,  in  the  sense  that  we  set  r)j"  (•)  =  1  in  the  generator  (2.2.1).  The 
corresponding  quasipotential  satisfies  the  Hamilton- Jacobi- Bellman  equation 


*=1 


dxj  dxi  J 

V  (-  -  - 

\  3’  3’  3 


0,  forxeS\{  ( 

’  n  1  3’  3’  3 


=  0, 


which  reduces  to 


x\  (eDxiv  +  eDxzv)  +  x2  (eDxtv  DxH  +  e  Dxi v) 
+  (1  -  Xi  —  x2)  (<n"  v  n'-v  +  e~Dxzv)  -2  =  0 

by  taking  the  affine  map  that  maps  S  to  G. 


The  quasipotential  of  this  problem  admits  an  explicit  solution  ( see  also  (5.9)  of 

mi) 

3 

V  (x1,x2,x3)  =  ^XilogXi  +  log  3. 

1=1 

V  is  a  smooth  function  on  any  compact  subset  of  infiS).  Its  derivative  became 
unbounded  (has  logarithmic  singularities)  when  approaching  the  boundary.  As  can  be 
seen  from  the  numerical  experiment  below,  the  highest  error  occurs  on  the  boundary. 

Let  G\  =  {x\  >  0.2,  x2  >  0.2,  x\  +  x2  <  0.8}  be  a  compact  subset  of  G.  Let  Gh  = 
G  n  hT2  and  G1)  =  G\  fl  hT2 ,  where  T2  is  the  two  dimensional  triangular  lattice.  Set 
M  =  99.  The  numerical  results  are  shown  in  the  table  below. 
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Table  4.1:  Maximal  error  table  of  Example  4.5.1 


n  =  1/h 

Max  Error  in  G J 

Max  Error  in  Gh 

10 

0.06 

0.1512 

20 

0.0306 

0.1160 

30 

0.0205 

0.0927 

40 

0.0155 

0.0772 

Figure  4.3:  The  graph  of  Vh  for  h  =  1/40  of  Example  4.5.1 
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Figure  4.4:  Numerical  errors  for  h  =  1/50  of  Example  4.5.1 

Example  4.5.2.  We  now  study  the  Glauber  dynamics  of  Curie- Weiss- Potts  model, 
described  in  Example  2-4-2. 


It  is  known  that  the  Curie- Weiss- Potts  model  has  a  phase  transition  behavior. 
There  exists  j3c  <  oo  ,  called  the  critical  inverse  temperature,  such  that  for  (3  <  (3C,  the 
Markov  process  defined  by  (2.2.1)  has  a  unique  stationary  distribution  at  5^ i/3,i/3,i/3)- 
For  d  =  3,  (3C  =  41og2  (see  [8]).  In  other  words,  (1/3, 1/3, 1/3)  is  the  unique 
equilibrium  point  of  (4- 1-1).  In  this  temperature  regime,  the  associated  quasipotential 
satisfies  the  Hamilton- Jacobi- Bellman  equation 


E  E 

i=l  j=l,j^i 


Xj  exp  (-(dxj) 

ELl  exP  (-/fofc) 


E(III 

3’  3’  3 


0,  x  G  <S\ 


1  1  1 
3’ 3’ 3 


0. 


We  study  numerical  experiments  for  (3  =  0.1,  and  take  x*  =  (0.2,  0.2,  0.6)  to  be 
a  representative  point  in  S.  Also  set  M  =  99.  Some  numerical  results  are  shown  in 
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Table  4.2:  The  value  of  Vh  of  Example  4.5.2 


n  —  l/h 

Vh  (: X *) 

10 

0.2157 

20 

0.1854 

30 

0.1749 

40 

0.1697 

Figure  4.5:  The  graph  of  Vh  for  h  =  1/40  of  Example  4.5.2 


the  table  below. 
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Appendix  A 


The  Condition  of  Schwartz  and 


Weiss 
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In  [35]  a  class  of  jump  Markov  process  with  jump  rates  vanishing  at  the  boundary 
of  some  set  G  C  is  studied.  We  now  formulate  their  conditions  in  our  settings. 


Let  G  C  Md_1  be  the  image  of  S  under  the  affine  map  (27,  ...,Xd~ i,  1— ^ 
(27, ...,  Xd-i)-  Then  G  is  convex  and  compact.  By  Lemma  1  of  [35]  G  satisfies  an 
interior  cone  property  with  parameters  (e,  /3):  for  any  x  G  d G,  there  exists  u  G 
such  that  for  each  t  G  (0,  e),  one  has  B  {x  +  tu,j3t)  C  G.  Also,  G  can  be  covered 
by  a  finite  number  of  open  balls,  and  the  balls  can  be  chosen  to  be  either  centered 
on  the  boundary,  or  having  no  intersection  with  the  boundary.  Let  {Bfj  be  such  a 
finite  cover,  Lemma  1  of  [35]  shows  that  one  can  fix  the  vectors  u  of  the  interior  cone 
condition  to  be  some  constant  Ui  in  each  region  B.L.  Moreover,  there  exists  7  such 
that  if  d  ( x ,  d G)  <  7,  then  d  (x  +  tui,  d G)  is  monotone  increasing  for  t  G  [0,  e]. 


In  [35]  they  study  the  jump  Markov  process  with  generator 


£n  (/)  (x)  =  n^Xv 


X 


vev 


f  (x  +  -v)  -  f  (x) 
n 


Recall  the  definition  of  C  {uj}  in  (3.5.3)  as  the  cone  generated  by  {uj},  and  denote 


Cx  =  C  {v  :  v  G  14}  • 


In  [35]  the  following  condition  is  assumed  for  the  proof  of  LDP. 

Condition  A. 0.3.  The  rates  and  jump  directions  satisfy  the  following. 

A.  There  exists  a  constant  K\  such  that  for  all  v  G  V,  |A„  (x)  —  \v  (y)|  < 
K\\\x  —  y\\.  Moreover,  the  rates  can  be  extended  to  a  5  >  0  neighborhood  of  G, 
so  that  the  Lipschitz  property  still  holds. 
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B.  For  each  x  G  d G,  there  is  an  e i  >  0  so  that  y  G  Cx  together  with  ||i/||  <  £i 
implies  x  +  y  G  G. 

C.  \v  ( x )  >  0  for  all  v  G  V  and  x  G  G° .  Moreover,  7,  e,  w*  of  the  interior  cone 
condition  can  be  chosen  so  that 


Ui  G  C  <  v  :  inf  A,,  (x)  >  7 

(  x&Bi 

and  if  x  G  £>*,  d  (x,  dG)  <  7  and  \v  (x)  <  7,  t/ien  A,,  (x  +  tuf)  is  strictly  monotone 
increasing  for  t  G  (0,  e). 

D.  C{v  :  v  G  V}  =  Rd-\ 


We  provide  a  simple  example  of  a  mean  field  interacting  particle  system  with 
K  —  1,  d  —  3  which  does  not  satisfy  the  strcitly  increasing  property  in  Condition 
A.O.3.C.  The  following  example  is  a  natural  3  state  particle  system  with  a  constant 
rate  of  one  for  transitions  1  — >  2,  2  — >  3,  3  — >  1. 

Example  A. 0.4.  Let  d  =  3  and  (•)}  specified  by  a  12  (x)  =  27,  0:23  (x)  =  x-i, 
a3i  (x)  =  X3,  and  zero  otherwise.  Then  G  =  {(x,  y)  :  x  G  [0, 1]  ,  y  G  [0, 1] ,  x  +  y  <  1} 
and  V  =  {ei,-e2,e2  -  ei},  Aei  (x)  =  1  -  xi  -  x2,  A_e2  (x)  =  x2,  A62_ei  (x)  = 
27.  For  x  =  (1,0)  ,  {v  :  inf ;ce b,  K-  (x)  >  7}  =  {e2  —  ei},  and  by  moving  along  any 
Ui  G  C  {e2  —  e  1},  Aei  (x  +  tuf)  =  0,  which  contradicts  the  monotone  increasing  of 
Aei  (x  +  tuf)  in  t.  Similar  results  hold  for  x  =  (0, 1)  and  x  =  (0,  0). 
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Proof  of  Theorem  3.3.6 
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Recall  that  h  maps  a  controlled  PRM  into  a  controlled  process,  and  is  defined  in 
(3.3.6).  Recall  also  the  definitions  of  and  A^^  in  Definitions  3.3.5  and  3.3.2, 
respectively.  The  claim  of  Theorem  3.3.6  is  essentially  that  the  additional  dependence 
of  controls  in  on  the  “type”  of  jump  is  not  needed,  and  that  the  variational 

representation  is  valid  with  the  simpler  controls  A^'' .  We  recall  that  the  controls  in 
,4®^  modulate  the  intensity  of  the  driving  PRM  in  an  s,  x  and  ca  dependent  fashion, 
while  the  controls  in  A^"1  multiply  the  jump  rates  A”  in  an  s  and  u>  dependent  way. 


The  proof  of  Theorem  3.3.6  will  follow  from  Lemma  3.3.4,  Lemma  B.0.8  and 
Corollary  B.0.7  below.  We  start  with  two  lemmas  which  elucidate  the  relation  be¬ 
tween  elements  of  A^V^  and  A^V^ . 


Lemma  B.0.5.  There  exists  a  map  ©n  :  A^V^  — »  A^V^  x  D  ([0, 1]  :  S)  x  A^V^  which 
takes  ip  E  A® into  a  triple  (dn,  /tn,  pn),  such  that  for  any  v  E  V 
1.  a™  (s)  =  f0Av(f*  (s))  <pv  (s,  x )  dx 


2.  p™  (s,x)  -  A"(A"S(i))  1[°P?(Ari(s))]  (x)  + 

3.  fin  =  h(^Nn^,pn( 0),An). 


Note  that  given  any  control  p 


E  A 


®|V| 


this  lemma  identifies  a  structurally  simpler 


control  pn  E  AfV^. 


Proof.  We  prove  the  claim  by  a  recursive  construction. 

1.  To  begin  the  recursion  set  to  =  0,  and  given  any  p  E  A define  for  s  >  to 
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and  v  G  V, 


An’°  {s) 

(s) 

W0  (s,  x) 

In  other  words,  for  s  >  0  and  x  inside  the  compact  set  [0,  A”  (/fn,0(s))],  we  set  </?"’°  to 
be  the  average  of  tpv  (s,  •)  over  the  set,  while  for  x  in  the  complement  we  set  =  1. 
We  see  that  by  construction  ||(^",0||oo  <  Halloo  V  1- 

2.  Assume  now  that  for  some  k  G  N0,  4  is  well  defined, 

(s)},  {a™’k  (s)},  {jj,n’k  (s)})  is  well  defined  for  s  G  [0, 1],  and 


^  (0) , 
r\  S(A"'°(»)) 


ipv  (s,  x)  dx, 


(s) 


An  (/tn’°(s))1[0’A"(/iri’0(s))]  ^  +  1[(W(£"-0W)]e  ( x ) 


n,k 

v  00,[£fc,00) 


sup  \(p™,k  (s,x)|  <  Halloo  VI. 


(s,:r)E[£fc,oo)  xR 


For  any  t  >  tk  and  v  G  V,  define 


Bk,v  (t)  =  {(s,  x,  r)  :  s  G  [4,  l],iG  [0,  A£(/tn,fc(s))]  ,  r  G  [0,  ip"’k  (s,  x)]  } 


and 

4+i  =  inf  ji  >  tk  such  that  for  some  v  G  V,  N™(Bk,v  (t))  >  o|  A  1. 

We  define  /tn,fe+1  on  [0,1]  by  setting  /jn,fc+1  (s)  =  (s)  for  s  G  [0, 4+i]-  At  4+i, 

update  iin'k+]  by  setting 


/T’fe+1(4+i)  =  An'fc(4) 


-  r,n>k 


2>/  / 

„£+  7  [tfc7fc+l) 


1[o,Aj(An-fc(s))]^^ 


1 


[0,oo) 
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and  set  /F,fc+1  (s)  =  /F,fc+1  (tk+ x)  for  s  G  [tk+ 1, 1].  Define 


a?:k+1(s)  = 


ffik+1(s,x)  = 


-A  ™(£™'fc+1(s)) 


^  (s,  x )  dx, 


d?’k+1  is 


\n  (/fH,fc+l(s))1[0,AJ(An’fe+1(s))]  (X)  +  1[o,A™(A”,fc+1(s))]C 


(X 


We  also  have  ||^’fe+1  |Li[tfc+1>oo)  <  II^IL  v  L 

3.  Recall  M'  defined  as  in  (3.3.5).  Since  IV"  has  a.s.  finitely  many  atoms  on 
[0.1]  x  [0  ,M']  x  [0,  the  construction  will  produce  functions  defined  on  [0, 1] 

in  L  <  00  steps.  Then  set 


A  (s)  =  F*  (s) ,  dnv  (s)  =  dnv’L  (s) ,  rv  (S)  =  (s) ,  if  s  G  [0, 1], 


Then  (pn  G  Furthermore,  by  construction 


/F  =  h  —Nnvn,nn  (0) ,  Ar 
n 


□ 


The  next  lemma  shows  that  from  controls  in  we  can  produce  corresponding 
controls  in  V40lvl. 

Lemma  B.0.6.  There  exists  a  map  En  :  — ■>  D  ([0, 1]  :  5)  x  ,4®lvl  which  takes 

a  G  into  a  pair  ( p,n ,  <p),  such  that  fin  =  h  (^iVni?,  pn  (0) ,  An) ,  where  for  v  G  V, 

Tv  (s,x)  =  A«(A"S(s))  1[°>A?(An(s))]  (X)  +  VajK/^OO)]^)- 


159 

145 

Proof.  1.  Define  to  =  0  and  for  any  a  G  s  >t0  and  v  G  V,  define 

/^°(s)  =  nn  (0) , 

(s,  x)  =  ^-^j^yy1[°,A™(/i".0(s))]  (a:)  +  l[o,A«(£"-°(s))]c  ix)  ■ 

2.  Assume  now  that  for  some  k  G  N,  tk  is  well  defined,  and  that 
(jLn,k  (s) ,  {<pk  (s)})  is  well  defined  for  s  >  tk .  For  any  t  >  tk,  define 

Ak,v  ( t )  =  {(s,  x,  r):sG  [4,  t]  ,  x  G  [0,  (/in,fc(s))]  ,  r  G  [0,  (pkv  (s,  x)]  } 

and 

4+i  =  inf  {t  >  tk  such  that  for  some  v  G  V,  Nf  ( A^v  (t))  >  0}  A  1. 

We  define  //n,fc+1  on  [0,1]  by  setting  jin-k+x  (s)  =  pn’k  (s)  for  s  G  [0, 4+i]-  At  4+i, 
update  /2n,fc+1  by 

/in’fc+1  (4+0  =  h”’fc(4) 

+  I Zv  [  I  I  ![o ms,x)](r)\K(dsdxdr), 

J[tk,tk+1)Jy  [,v^  W'J  J[ o,oo)  n 

and  set  /ir':fc+1  (s)  =  pn'-k+l  (tk+ 1)  for  s  G  [4+i,  1]-  Define 

94  (S,  X)  =  ^  (pn,k+l(s^  1[0,A™(An-fc+1(s))]  W  +  1[0,A«(/i^+1(S))]C  W  ' 

3.  Since  Nf  has  a.s.  finitely  many  atoms  on  [0, 1]  x  [0,  Ha+HoJ,  the  construction 
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will  produce  functions  defined  on  [0, 1]  in  L  <  oo  steps.  Then  set 


h”  0)  =  ( s ) ,  <pv  ( s )  =  ^(s),s£  [0, 1]. 


Note  that 

(s,  x) 

and  fin  satisfies 


fin  =  h  i_in  (0) ,  An^j  . 


□ 


The  next  result  is  a  corollary  to  the  construction  in  Lemma  B.0.6. 

Corollary  B.0.7.  For  any  {d„}  G  given  and  t  G  [0, 1]7  S”  (a)  (f)  (defined  in 

Lemma  B.0.6)  has  the  same  distribution  as  A”  (a,//1  (0))  (t),  where  A”  is  as  defined 
in  (3.3.9). 

Proof.  Recall  that  fin  =  Ef  (a).  We  have  S”  (a)  (0)  =  /in  (0).  Given  s  G  [0, 1],  the 
total  jump  intensity  of  fin  (s)  in  the  direction  v  is 

/•A ?(/!"(»))  _  rA?(An(A)  &  fs\ 

l  =  I 

which  is  the  same  as  that  of  A”  (a,  /in  (0))  (s).  □ 

Lemma  B.0.8.  Let  Ab,  Ab  and  A  be  as  defined  in  Definition  3.3.5,  Definition  3.3.2 
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and  Definition  3.3.1  respectively.  Then  for  F  G  Mb  (S), 


inf  E 

^f|v| 


=  inf  E 

<^e.4®lvi 


=  inf  E 

ae^f|v| 


27  LiW  +  HP")  ■  P*  =  h  (0) ,  A”) 

vGV  ' 

E  Mp.)  +  f (pn) :  P"  =  h  (1  rr*,  f  (o) ,  xA 

.vGV  '  ' 

E  f  K  (Pn(t)) 

.vev 


av(t) 


A-  (/2"(t)) 


dt  +  F(fLn)  :  /T  =  S”  (a) 


where  En  is  as  defined  in  Lemma  B.0.6. 


Proof.  The  first  equality  is  a  consequence  of  Theorem  2.4  of  [5].  To  prove  the  rest 
of  the  claim,  for  a  G  fixed,  let  (p,n,  (p)  =  Sn  (a).  Then  by  definition  p  G  .40lvl, 

and 

E  f  \  t(<Pv(t,x))dtdx  =  ^2  [  (fin(f))  t  (  dt ■ 

veVJo  Jo  vevJo  WDTP))/ 

Now  it  follows  from  Lemma  B.0.6  that 


inf 

(p£A®  lyl 


<  E 

=  E 


E  LiM  +  F(fi")  :  pn  =  h  (1  N"*,  f  (0) ,  A  A 

.vEV  ' 

E  MftO  +  F  O  h  (Itr*,  p"  (0) ,  A”  j 

dt  +  F(jT 


E  /  K(fin(t))t 

Iv&V 


a„(t) 

Wfj) 


The  reverse  inequality  is  proved  by  a  convexity  argument.  Recall  the  definition  of 
0”  given  in  Lemma  B.0.5.  For  given  ip  G  let  (a,  ft!1)  =  (0”  (tp) ,  0”  (p)).  Then 
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a  G  By  convexity  of  d  (•)  and  Jensen’s  inequality, 


i  ( ipv  (s,  x))  dx  j  ds 
tpv  (s,  x)  otaA  ds 


=  7.  kmkkm) 


Summing  over  v  G  V,  applying  Lemma  B.0.6  and  infimizing  over  a  G  we 

obtain  the  desired  result.  □ 
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Appendix  C 


Proof  of  Theorem  3.4.1 
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Proof.  The  idea  is  to  use  uniformization  to  represent  Xn  —  Yn  as  a  jump  process  with 
(well  controlled)  small  jump  rates.  Since  A"  (•) ,  A„  (•)  are  bounded  by  some  constant 
C,  let  7=  |V|  (C  +  1).  Also  denote  q  ( x )  =  ^ev  A„  (x)  and  q 71  ( x )  =  A”  (x). 

We  then  define  the  uniformized  transition  matrices  on  Sn 


P 


x,  x  H — v 
n 


pn 


x,  x  H — v 
n 


K  (x) 
7 


,P(x,x) 


q(x) 

7 


K  (x) 

7 


(x,  x) 


qn  (x) 
7 


And  let  Xn,  Yn  be  the  Markov  chain  on  Sn  starting  at  x,  and  with  transition  matrix 
p  pn  reSpectively.  Let  Nt  be  a  Poisson  process  with  rate  7,  and  is  independent  of 
Xf  and  Ytn.  Then  Xjtj  ,Y$  has  the  same  distribution  as  A”,  Yf1,  respectively.  It 
then  suffices  to  couple  Xn  and  Yn  so  that  their  displacement  is  a  Markov  chain  with 
small  transition  rate. 


We  can  construct  a  coupling  (Xn,Fn)  of  Xn  and  Yn  as  follows.  We  define  a 
Markov  chain  on  Sn  x  Sn  that  starts  at  (x,  x)  and  with  transition  probability  that 
for  each  v  G  V, 


{x,y) 

(x,y) 

(x,y) 


(  1  1 

x  H — v,  y  H — v 

\  n  n 


x  H — v,  y 
n 


x,  y  H — v 
n 


J  with  probability  min  {A„  ( x ) ,  A”  (y)}  /y 
with  probability  (A„  (x)  —  A”  ( y))+  /y 
with  probability  (A„  ( x )  —  A”  (y))~  / 7 


and  stay  at  ( x ,  y)  otherwise.  Let  Zn  =  Xn  —  Yn,  then  Zn  (0)  =  0,  and  it  jumps  in 
some  direction  v  E  V  with  probability  at  most 

|A„  (x)  —  A"  (■ y)\  /y  <  (C  ||a;  —  y\\  +  O  (1  /n))  /y,  if  the  current  position  is  x  —  y. 
Thus  Xf  —  Ytn  has  the  same  distribution  as  Zf=Zf  ,  which  is  a  Markov  process 
starts  at  0  and  jumps  in  some  direction  v  E  V  with  rates  at  most  C\\x  —  y\\  + 
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0(l/n)  for  some  C  <  oo.  And  it  sufficess  to  get  an  superexponential  bound  for 
P(suPte[o,i]  ll^n(*)ll  >  <*)■ 


If  Zn  were  on  one  dimensional  lattice  this  bound  is  implied  by  Theorem 
3.6.1.  Let  Wn  be  a  birth  process  on  which  jumps  to  the  right  neighbor  with  rate 
Qx,x+i/n  —  Cx  +  C2/n,  for  some  C2  large  enough,  and  stay  at  x  otherwise.  By  the 
comparison  principle  of  ODE,  it  follows  that  Pid  (. Zn  (t)  >  c)  <  Pi^  (Wn  (t)  >  c)  for 
any  c  >  0  and  t  G  [0,1].  Thus 


P  Id 


sup  I Zn  (t)\>  8 
te[  0,1] 


< 


P  Id 


sup  \Wn{t)\  >  8 
te[o,i] 


<  P1(i  {\Wn  (1)|  >  <5) 


v  >  j= 1 

fC\Sn  f C2/C  +  8n\ 

U;  v  c2/c  J 

/  r<\  Sn 

<  (-)  {C2/C  +  8nf2/C 


which  goes  to  0  superexponentially  as  n  — >  00. 


The  general  result  follows  from  one  dimensional  path  counting.  For  n  sufficiently 
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large,  we  have 


< 

< 


< 


< 


< 


P  sup  \\Zn(t)\\  >  8 
\te  [o,i] 

P  (Zn  jumps  at  least  8n  lattice  steps  in  [0, 1]) 


P  ( Zn  jumps  k  lattice  steps  in  [0, 1]) 

k=Sn 


oo 

#  {lattice  path  in  Sn  with  length  k}  ■  Pid 

k=Sn 

oo 

^  (2 d)k[C/n]k(k  +  C2/Cf2/c 

k=Sn 


^  (2 d)k  [C ln}k/ 2 

k=Sn 


[Ad2C/n]Sn/2- 


1 

Ad2C/n 


sup  |  Zn  (t)|  >  k/n 
te[  o,i] 


where  the  last  inequality  follows  from  the  fact  that  [C/n]k^2  ( k  +  C2fC)C '2//C  <  1  for 
n  sufficiently  large.  Therefore 


— — logpf  sup  ||Zn(£)||  >5 
n  ye[o,i] 

-  (yiog^C'/n]  +C3) 

£ 

>  -  log  n  +  C'4, 


for  some  C3,  C4  >  0.  The  expression  goes  to  00  as  n 


00. 


□ 
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